Software Engineering in the Age of AI: Full Training Transcript
Good morning. Good morning. Good morning. Welcome to today’s training. We’re going to be talking about software engineering in the age of artificial intelligence.
And I’m going to frame this as an engineering analysis, not hype about AI. Um, our our industry has been through a lot of transformations and we’re and this is this is another one. So um if this is your first if this is your first training my name is Jeffrey Polarmo and I’m the chief technology officer at Clear Measure and our mission is to empower software delivery. Our vision is to in improve and inspire software teams worldwide and um and and for the last year we have been doing uh AI implementations, AIdriven development uh training, coaching, advisement and so if uh if you have a project where you are looking to get the benefit from AIdriven development then uh perhaps we can help. We are pretty easy to find.
Reach out. Um so so let’s get let’s get into this. Um our our industry has been through a lot of uh transformations and the a transformation is is it’s qualitatively different. It’s a new paradigm for user and computer interaction. It’s not just another tool.
Um, and so as we go through this, we’re going to talk about the technical implications, architectural considerations, and process changes for integrating AI into our software delivery. So over over the decades, um, you we’ve we’ve had the introduction, and this is just in my career, we’ve had plenty more, but um, I started programming when the internet was not widespread. Um, I was I was uh I was very weird because I had this thing called an email address. Um, but but we’ve had the internet transformation, we’ve had agile, we’ve had DevOps, we’ve had cloud, we’ve had mobile. And of course, mobile and cloud kind of overlap, not sequential, and now we have AI.
And each of the prior transitions has shaped how we build software and how we deliver software, how we run software. And AI is a change in kind, not just a change in degree. And um the the constraints that we’ve dealt with over time move from actually writing the code, you know, typing characters to validating the code to know what we have is good because now the machines can can uh can write code for us. So uh you know every everything else really follows from that. Okay.
And all right let me go to the next next slide. All right the question answer asymmetry. I want to I want to kind of deviate a little bit and talk about the scientific method. So in most people in grade school learn about uh the the scientific method where the first thing you do is you come up with a hypothesis and then you derive experiments and then you run experiments and you see if your hypothesis was true and you iterate. Uh the scientific method is is very useful.
However, uh even in the scientific community and I and I’ve searched um there is no available training method or logical progression that we have available in this world to teach upand cominging scientists how the heck to come up with the starting hypothesis. Now once we have a hypothesis, well now we have the scientific method to evaluate that hypothesis. But how does one come up with a hypothesis? There’s not a method. You can’t you can’t train someone in how to come up with brilliant hypothesis.
It’s just not out there. Uh how did how did Albert Einstein come up with the theory of relativ with the hypothesis that caused him to start running experiments that ended up being the the theory of relativity. And um it’s just not out there. And I I I challenge you to go find it. In the same way, these these language models are highly proficient at generating responses to well-formed questions, to well-formed prompts, but they lack hypothesis generation.
They can’t formulate meaningful research questions, architecture questions. They can’t decide what problems merit investigation. And and uh so this is the same thing as coming up with a hypothesis. Language models and these AI tools can answer questions and they can respond but they cannot figure out a useful question to ask. That is software architecture and that is software engineering in the age of AI.
We have this supernatural creative brilliance where we’ll be could be driving in the car, we taking a shower, going to the grocery store, and then boom, some inspiration will come into our mind and then we’ll say, “Aha, I need to investigate X or Y or Z.” And now, now that I have a question, now that I have a hypothesis, great. These AI tools can help me research, can help me investigate, but the language models do have no understanding of of there’s just that’s not that that’s a that’s a unique part of our intelligence. Okay. All right. So, um there’s also a few realities of these language models.
Um, and these models they they operate by using probabilities and mathematical weights to traverse a neural network of digital neurons guided by our system prompts and our user inputs and the tool results. But there’s actually no semantic awareness. There’s no genuine understanding of what any of that instruction actually means. There’s no way for a language model to know that something is complete. It it can’t determine when a task is actually done.
There’s no quality assessment, no intrinsic mechanism to evaluate output quality. And and so we have to remember that these language models and the AI harnesses that wrap them, they cannot selfassess. So quality has to be judged externally. That’s a big idea. Quality has to be judged externally.
They can’t evaluate their own output. They and of course all of us at this point have been been prompting language models to know that we get back trash a lot of the times. And this language model will generate some text trying to deceive us as if there’s some awareness of the task at hand. And the text sounds confident, but it’s just baloney. So the models can’t evaluate the tool can’t evaluate its own output.
Validation has to come from external systems. We have to have an external feedback loop. There’s there’s uh error signals from compilers. That’s from the outside test frameworks running automated tests the build systems or integration build that all has to feed back into the context window and then it’s possible for an iterative refinement process because the definition of quality comes from the outside and so we have to stand up comprehensive external standards. Yes, the code obviously needs to compile.
We have some linting tools. We want to run our tests and and running those tests and executing them well a test failure that’s that’s valuable. There’s also performance benchmarks and security compliance uh tools. There’s architectural consistency checks that we can run but quality has to come from the outside. Okay.
All right. So in the age of AIdriven development, we shift the primary constraint from code generation to validation. The traditional bottleneck was implementation speed. Are you done yet? Do you have it working yet?
Whereas AIdriven development creates new bottlenecks. It shifts the bottleneck into verification, quality assurance, that part of the software development life cycle. And so validation pipelines, however you have it set up for for your team of your software, they now need to be faster than in the agile and DevOps and conbon era because changes can only move as fast as your ability to validate them. And there’s already stories of of just pull requests, mountains of pull requests, and the and the time it takes to validate each pull request. And of course, we we see what’s what’s natural when we’re drowning with a 100 pull requests and it’s my job to look at them.
Well, then I have to decide, do I trust do I trust do do I not do my review and trust the person who submitted the poll request because man, this is a really this is a really good guy on our team. He’s very senior. Let me just hit the button. Of course, this poll request is good. I don’t have time to look at it.
All right. So, so there’s that pressure, but it is true that the bottleneck is in validation. And so, if we’re going to move fast overall in our entire process, we can’t just move the bottleneck, but preserve the bottleneck. We have to break the bottleneck. And for the last 20 years, there’s been general guidance about the 10-minute build.
And a lot of significant software systems don’t actually have a 10-minute build. and still struggle to get it, you know, less than 20 minutes. But our integration build, we’ve kind of all known and and research has shown that if it’s if it’s too long, it won’t get run. It won’t get paid attention to and then the value goes away. And so we have an automated build and we think about how long it takes to run. Well, if if we have more and more cycles of branch commit pull request, then we are going to be running our build more and more and more.
And if we still have the same build speed that we’ve had for the last 20 years industrywide, we are not going to speed up as a team. And so the inner cycle time of your build has to increase dramatically. Okay, you have to. Now the general guidance and process for a continuous integration era build is sequential. You do some environment setup.
You do a compilation. You do unit testing. You maybe build build up your development database. Then you do integration testing. Then you do uh release candidate packaging.
And then you push your release candidate package over to Octopus deploy or your re your uh automated releasing pipeline. And that’s your continuous integration uh process sequentially. And that takes you know that takes you however long it takes. Let’s say it’s 20 minutes. That’s too slow in the age of AIdriven development.
You have to make it faster. Now, how do you make it faster? Well, you tune everything. And one of the ways to tune the build as a whole is you make all of the processes run in parallel. Uh the testing and packaging and and compilation, the you you distribute the tests in multiple shards of execution.
You take advantage of running tests in parallel. Uh yes, you give your build servers more more hardware, more compute. The hosted build agents of Azure DevOps and GitHub actions um they are not fast enough uh for the age of AIdriven development. And now you can you can do your own self-hosted, you can do large, but I’m talking about the commodity ones that are that are free or provided with your account. And so um we can also for integration if we’re using a particular SQL server we can use a smaller set of data.
Uh we can use an inprocess database like SQLite um for for the inner loop. I mean there’s all kinds of things we have to figure out a way to get our our integration build faster because that has to be the mainstay of these pull request reviews. Now I’m going to show you a screenshot of uh a training app uh that we use in our AIdriven development boot camps which by the way if your team is interested in an AIdriven development boot camp specifically for software engineers and technical architects uh just just give us a call. We schedule them we we schedule them as our clients need them. Um, from time to time, uh, we do, uh, some public class trainings as well, and we put those on our website.
We’ve done maybe two over the last year, but if your team would like, uh, a dedicated class, just let us know. And so, um, what I’m going to do is I’m going to put the, uh, I’m going to put the URL for this project in I’m going to put that in the chat window right here. So, for those following along live, you can see uh, you can see the just a second paste. You can see the URL. If you are watching the recording of this later on, just know that we do these trainings live and you can follow along.
So this is this is GitHub actions and you can see that it’s not sequential. We have all of these jobs running in parallel and these are all things that need to run but they’re in parallel. Now the total time is still 6 minutes and 48 seconds. So it’s not meeting that 5 minutes that I’m challenging to. But but you need to be stingy.
This your entire integration build has to be fast. And so you have to take every little thing and running it in parallel. So the next step here would be taking those acceptance tests uh the acceptance test at 6 minutes and 24 seconds and splitting it up into possibly three shards so that we can take a third of the test suite and and run it in parallel. Um so that’s the way you have to think. You have to tune your inner development loop.
You have to tune it. Okay. Now um let’s let’s also look at the type of code that is changing. So we’re talking about reviewing pull requests. The the bottleneck has moved to pull requests and now we need to figure out if whatever changes is in that pull request is good.
So obviously the build has to run but what if we broke the architecture? Let me give you an example. Let’s say let’s say that we have a skyscraper and sometime in the future there’s not only these AI tools that can move around code but we had this automagical 3D printer that can just automatically fabricate and configure literally an office the floor of an office building and it can like and it can create carpet and it can uh put window dressings and it can it can actually move non-loadbearing walls and move things around just like the AI tools you know, moving code around, moving window dressing, moving walls, those things. And this machine decorates the office and builds out uh builds out the floor of an office building, and it’s fantastic. And you know what?
If it gets the carpet wrong, no problem. We can just redo the carpet. However, if this automagical 3D printer machine moves on that floor where the stairwell is or where the elevator shaft is or where the pipes of the fire suppression sprinkler system is, well, now it has just broken the architecture of the building. It has just broken the structural integrity. The systems that move that integrate from floor to floor to floor.
Okay. Same thing in code. Code carries varying significance. Okay. There is structural code.
Those are core architectural problems, domain models, integration uh interfaces. Uh and those figuratively those are the steel girder beams and the foundations and the elevator shafts and how the how the water goes to the fire suppression sprinkler system and the and the fire escape stairs. Then there’s code that’s superficial. Those are UI components, formatting logic, utility functions, you know, decoration, window dressing, carpet, furniture, that kind of thing. Today’s AI tools show no ability to distinguish between window dressing code and structural code.
They just they treat everything like a change with equal consideration uh unless there is some context coming in to give them a clue. Okay. And so we put things in agent MD and co-pilot instructions and skills and various things to kind of help with that. But innately code is code. There’s no there’s no understanding of that.
And so when we evaluate a pull request, not only must the build work and all the tests work, we have to look at all kinds of other things about the change to know if something structural was just broken or if it changed something that we know is if we break the architecture then our our problems are going to happen a little bit down the line. And when I am handcrafting every line of code myself with my fingers and and my resharper, then I’m kind of evaluating the important code versus the structural code all along the way. And that’s how we’ve always done it. All right. So because AI can’t judge the significance we engineers have to run a risk proportional review uh for for lowrisk changes you know a single single method implementations UI adjustments the formatting uh you know if you get it wrong okay it’s super easy to redo it very low risk but the high-risisk changes the architectural pattern modifications your domain model changes the integration changes.
Uh we have to have a a higher risk review and so we have to put the reviewer’s attention where the blast radius is the largest. Every aspect of what it means to be a good change to the software has to be validated. And and as we find better ways to automate the verification of what it means to be good, uh then then we can shrink back the manual manual review. But the aspects that require that manual review um it can’t be too big. We have to figure out ways to shrink it.
And as we go along this journey, all of the pull request has to be verified. And and so we will we have to find automated ways to verify different aspects of what it means to be good so that the surface area of that pull request that still requires my eyeballs, the surface area shrinks and shrinks just so that we can we can actually do a review and know that that change to the software is still a good one. So um a big idea here quality gates have to get stronger they also have to get faster. Uh the theory of constraints was put forth in a book from Eli Yahoo Goldrat named the goal uh in I believe 1982. It’s a mainstay of of NBA programs all over the place.
And right around the same time was the book about the Toyota production system and that was about flow and quality control and stop the line and and insist on absolute quality. Those those principles apply even more today in the age of in the age of AI uh than they have. And you know teams typically spend 3/4 of their time on either rework or coordination about work rather than the actual work you know which is new feature development and and AI just amplifies this problem. Um because there is more work going on and if and as as as work creates rework well well the amount of rework increases. So AI amplifies the problem unless unless quality gates get better and get faster.
Speed alone without stronger quality gates just produces defects faster, which increases the amount of work that the team has to do, which shrinks the piece of the pie, shrinks the portion of the week that the team is actually working on new work, which actually for the organization means the team is going slower at delivering requested work. Okay, that’s that a it’s a big big dynamic. All right, let me now let me show you how to evaluate. I’m going to I’m going to switch over to this URL that I showed you and I showed you in the um a folder called the codebase audit report. Now, if you look inside, if you look inside this repository, you will see a clawed skill for what’s called a codebase cgraphy report.
And that is basically, hey, look at this code and and figure a bunch of stuff out. And so, it’s running uh the SEC tool for lines of code and complexity. It’s running Roslinator. It’s evaluating tests. Uh it’s running some custom some custom C scripts.
And it’s generating architecture diagrams so that we can reason about the shape. And and so I’m going to I’m going to fast forward through these diagrams. I’ll come through. But one of the things one of the things that you want to know with any any codebase is well what’s in here? How many files of the certain type?
How many lines of code do we have? And so you want to know that. Okay. Um and then per component cyclomatic complexity. This is an easy gate.
If you look at your average cyclomatic complexity per line of code or per file and you just increased it beyond a percentage uh then you know what maybe that pull request that came in maybe you just uh degraded your software. Um and so this is more about cyclomatic complexity linting. Also um uh this particular one does a quantified size analysis using the international function point user group uh definition of a function point which is still in our industry the only way to economically size how much software we have. That’s the only measure we have and very few few people are trained on the method for how to do it. But guess what?
An AI tool can do it. How about that? So, so now you can actually now you can actually compare the size of your software system with the size of all the other systems that are in our body of research and capers Jones uh his books are probably the the most well-published uh body of research around software and then based on your software size you should know your team’s productivity and and all that. Okay. And then um and so an economic valuation if you are if you are a uh if you’re a conservative team you know what to build that amount of software it might cost you this much money but if you are an elite modern team then that same piece of software would only cost you this amount of money.
And yes at Clear Measure we have evaluated the different methods that yield uh different paces of of software teams. And so um uh when we when we do a project we are reliably at the at the at the aggressive level. Okay. Uh but but we help we help our clients uh you know anywhere on the spectrum get up to this aggressive level. All right.
And then there’s findings and and that is okay. Here’s here’s a bunch of problems with the software. Now let me put this away because that’s a lot of information and let me just go through and some of the diagrams that we can pull out that we can look at in our pull request if we want to automate these diagrams. This is one of them. This is our this is our onion architecture.
This is the logical shape of the codebase that we have in front of us. And you can see in the middle we have our core project. the these are these are are domain core and uh while while the database is there because the database project is just SQL scripts it doesn’t reference anything nothing references it so you see there’s no actual arrows coming into it um it’s just like a a side project there and then we have other projects that reference core and other projects that reference those projects and then you can see the different rings of the onion now the the technical plant UML um technology it doesn’t Actually I don’t know how to render circles. So we have you know concentric rectangles but this is our onion architecture. So we can see where in the different layers uh different projects go and of course we know that the direction of dependency needs to go towards the center. And if we have a core project, a core assembly with all the important stuff that needs to be stable, we know that it can’t reference other code that is less stable than itself or it just that kind of breaks the architecture.
So we we can literally harvest our onion architecture diagram from the code. Um and and when when clients hire us to do an audit or just advise them and come in and help, this is one of the first things we do. We take their codebase and we learn about it. And so we go into a codebase that we’ve never seen before and we start deriving understanding so that we can start understanding what the shape of this software is and how it works. And so you can put some of these in your pull request validation process so that you get early signals if something accidentally deviated from the structure that you wanted.
Um, and here’s a logical view annotated with lines of code. These are your you these are your projects your assemblies and the system code this big box well that’s what that’s different from your DevOps code because we don’t deploy DevOps and test code to production we deploy the system to production so we need to know how many lines of code what our cyclomatic complexity what type of code it is um and uh be able to reason about it and so that’s another that’s another good tool so that we can see the relative lines of code and this particular codebase this has a problem. How do I know? Well, let me look at the UI. Let me look at uh UI client UI shared.
We’ve got what upwards of uh four 5,000 lines of code up here in the UI. Let’s go into core. Why does core only have 1,87? Core should be should be our biggest. Okay.
And so there’s opportunity for looking around to see now do we have a whole lot of pages, a whole lot of markup maybe, but it’s a signal that we need to look at this to see if logic and behavior and rules is accidentally being put in the user interface instead of into our domain model. Okay, so that’s a signal. That’s a quality signal in this particular codebase. All right. Uh and then this is our testing pyramid.
So we take our take our uh testing libraries, unit tests, integration tests, and acceptance tests. And look at the number of tests. Do we actually have a pyramid? Do we have more unit tests and integration tests? Do we have more integration tests than acceptance tests?
Um and in this case, we do. So we have an actual pyramid. So that’s great. But we need to we need to look at this because um you know it’s easy it’s easy for the AI tools to uh to not not keep our testing pyramid. All right.
So those are some of the diagrams that are in there. I invite you to look at that URL uh in the link. Okay. So, um, I want to go against the grain a little bit with people out there in the industry that are calling AI a team member or a teammate. It is a worker.
It is a it is a piece of equipment that you want to leverage and you want to keep the piece of equipment going. um in a factory. We’ve kind of learned that we have specialized equipment in all kinds of factories. And if the equipment is idle for long periods of time, then it’s not useful. And in a lot of factories, there are certain there are certain machines that are the bottleneck that are the lynch pins in the factory. And so everything is optimized around getting as much as as much material processed through that machine as possible.
So that machine needs to keep working and work in the most efficient manner possible. It is still a machine. If the power goes out, it it it stops running. If you starve it from inventory input, it stops working and it can’t do anything unless an expert is operating it. AI tools function as as that a great piece of equipment that requires a skilled operator.
It’s not an autonomous um entity. The word agent is is confusing because when we apply the word agent, we’ve long we’ve long had uh customer service representatives and then we rename them rename them to customer service agent. We have insurance agents. When we label someone with the term agent, we are we are we are saying that that person has agency, that they have some awareness, some authority, and one that they’re a person. And so labeling something as an agent kind of deceives us into thinking that this machine or this tool is somehow going to be aware of something and somehow do something and take initiative or have agency but it’s just a piece of software.
Okay, we we have if we understand the architecture of the language models and the architecture of the AI tool harnesses that are operating these language models, it’ll help us put it in its place. So, it is a piece of equipment. It’s not a team member. Now, it’s an amazing piece of equipment, but it’s not a team member. So uh in in police in the military there are dogs there are canine units and and they have a specialized capability under expert guidance.
Even even a dog a dog is you know intelligent. You can train a dog to do certain things. Uh um a dog can even kind of show emotion. All a dog is is an is is amazing. But the dog, as amazing as it is, still has an operator.
Okay. Um, so you got agricultural machinery. Well, those are force multipliers big time. Combines, tractors, the the the d the uh uh adaptive the the cruise control uh things on some of these big uh big piece of farm equipment is pretty amazing. They’re pretty autonomous these days on the big end. uh but they are still machines and and the effectiveness of all these machines AI tools included correlates directly with the operator competency.
So we software engineers we solve or or we um we translate ambiguous requirements into solutions. And a lot of these a lot of these uh uh ambiguous problems they have multiple possible solutions and and we have to come up with what an appropriate solution is. And once we’ve figured out the right solution, okay, now the programming work comes in and then as programmers we implement the tasks and the and the work that has been broken down of the chosen solution. So yeah, the AI tools can do the programming. Now once we have a solution and yes we can even use it to do research to help us come up with the right solution but we have to start the process asking the right question coming up with a hypothesis and then evaluating the possibilities and choose a direction and then yes the programming is largely automated in in a lot of cases and so AI amplifies our existing capability and our existing skills it it is not a substitute for engineer ing judgment or architectural thinking.
And so, uh, I just I wanted to kind of frame that because that’s how I that’s how I think about it. Now, uh, some of you have read the book called The Mythical Man Month. It’s it’s by it’s by a gentleman named Fred Brooks and, um, that is a series of essays and one of the essays in that book is called the surgical team. And I believe that it’s newly relevant and that is that small highly skilled teams can now match the output that once required larger larger teams and often two or three senior engineers with AI proficiency can can match the output of a team that we might have previously staffed with 10. So um let’s let’s uh let’s just look at some of the lowhanging fruit. time is saved looking at API documentation, researching, doing uh Stack Overflow searches.
Uh I mean that was very popular Google searches. Uh how to figure out how to use a library uh boilerplate code generation. And so uh we have a trade-off here. No longer is fighting with the syntax or figuring out how to use an API or library uh taking hours. But we have a trade-off.
We can have smaller teams now, but those teams have to be filled with highly individualized skill. Okay. Now, let’s think about the the uh surgical team. Think about there are in this in this structure there’s two surgeons. They ultimately know how the surgery is going to go and they perform the surgery.
But there’s other people in the room able to contribute to the health of the patient. There’s your IV drip bags, there’s your anesthesia, your blood tests, your uh prepping tools, there’s getting a swab, um there’s, you know, suction. And with AIdriven development, we have a great opportunity to involve more software stakeholders in the in the delivery of custom software beyond just those who intimately know how the code is working under the covers. I’m not suggesting that non-programmers are suddenly programmers or non-engineers are suddenly engineers, but I am suggesting that some lowhanging fruit scenarios are available to us. For example, an analyst can now with an AI tool prototype a change to an existing screen and pilot some options for user interaction in order to see which screen adjustments might work the best.
And the prototype might actually be on a throwaway code branch in Git. And it might prevent some extra cycles because the analyst is able to request from the software engineers in detail the change that he’s seen already with his own eyes. Then the engineers can implement the change in an appropriate way. Now if if the change is just changing a name display from first name last name to last name, first name, well well maybe the prototype branch and the correct implementation they might not be very far apart. So there’s a wide variety of possibilities here to include more people in the software delivery process and then have that two very skilled software engineers with the hand in the patient but have the other people in the operating room contributing in different ways that um that that drive the process forward.
Okay. So let me I want to I want to deviate real quick and show a quick demo. And what I’m going to do is pull open. Okay. So, I’m going to All right.
I have a work item on a conbon board. And I’m going to show you that this is a wide conbon board with a development column in the middle. And then we have some design and analysis steps, UX design, technical design, test design on the left, some validation steps on the right. All right, let me zoom back in. And I I have uh I have some automation in the background implementing our AI software factory pattern.
And I’m just going to label this as uh you know available for automation. All right. So we labeled that and I’m also going to take one that I have on the board here and I’m going to also move that over. All right. So, you you see that our automation actually already picked this up.
And um one one thing that we can do is we can pick apart different aspects of our work and we can actually um get our AI tools to do little bits of it. So, let me just see what’s going on with this guy. We see that uh this was this was the only thing. Change the footer to copyright 2027. Oftentimes, we had that.
And so, we did some analysis, user experience design, and actually got it doubled. That’s a little bug. But I’m going to click and over to a cursor session that actually did the UX design just now on the spot. and it looked around in the codebase and it figured out, you know, what what it needed to change. It found where uh it found where the copyright year was in the application. Okay.
Uh let’s go over to this one, an API endpoint. And uh I actually put it straight away. Actually, let’s see if we can do this one if it was on demand for time. Nope. I’m going to go to the other one and all right I have cursor now for for the test design I have a different tool I’m I have IBM Bob and so IBM another longtime tech company also has an AI harness and I’m going to open that and this is actually a terminal T-U session and this is now there’s no formatting here but this is IBM Bob processing it under the covers and I have a dynamic uh I have a tunnel going to it just on my box so that we can see that it’s working and it’s just processing language model things just like cursor it’s just cursor was on a web page and this is on a terminal uh but it’s but it’s doing the work under the covers and by the way the AI software factory is a pattern where you’re seeing what parts of our work are possible to automate so that we can take work away from our software engineers okay And and so one of these as this continues to run and I go back to the go back to the conbon board here is UX design technical design we can enrich we can enrich the work item if coming in someone just said hey we need to change the footer we need to say copyright 2026 or 2027 or whatever’s stupid little change here we can enrich the work item item text by investigating what’s there.
And if you think about it, if you’re going to manually process this, we found that um we found that a plan implement process pretty much is a good idea with most of these AI tools, right? Where you do a plan and then you implement. Why? Well, because the plan ends up being a pretty detailed prompt for the implementation step. Well, for simple stuff, the planning step, it doesn’t need to be done with me with hands on the keyboard waiting on the tool.
That thing can be automated. So, we’re automating the plan. All right. And uh uh and and then for test design, well, once it’s done, how are we going to test it? So this this particular thing with the API endpoint um if we if we we go back to our terminal session, it’s just automating and it looks like it’s just about done with the plan.
Um test data. Yep, there’s some it’s it’s it’s even naming some possible tests that need to be run. And it’s even a checklist. The checklist, you know, create this, implement that, create the test, verify all tests pass, the endpoint returns HTTP with JSON. And so these are all things to validate.
And then at the bottom, we’re going to Okay. So if if I’m if I’m doing the timing right, it should Yep. And it updated the test body while we were watching it. And there is the now a test design section with the scope out of scope. Here’s the integration tests that need to be implemented.
And then here’s a here’s some test. Here’s a format of JSON to return back. And here’s a checklist. And so for again this the scope of this was add a new API endpoint named this that returns JSON object with basic runtime metrics like uptime total rec. Now if our system doesn’t yet have any kind of concept to get uptime or total requests well then that has to be there.
But when this goes through, it’s after the domain model and after the system has these capabilities. And now all we’re doing is making it retrievable with this API. This isn’t complicated. And so if we’re going to do a plan, if I’m going to manually do a plan, well, guess what? This is what’s going to come out anyway.
So why not just automate that step? All right. So, let me go back to uh oh, and while while we’re uh while we’re doing, you know, individual things and and watching watching the terminal and watching our, you know, cursor sessions, we can even we can even see the cursor sessions that do that do the actual coding and change the actual files. Um, so we can we can watch all of these AI tools whether it be Cursor, Claude, IBM, Bob, C-Pilot, Codeex, Grock, um, any of these things and we can flow things um, through the system and just things will happen automatically and you can and in GitHub you use labels so that you can say that okay this type of thing is compatible with my automation and it fits and so let the automation spring into action versus other things are not labeled and and we say okay you know what that works work that work really does need to be done by an actual software engineer that that type of work is not yet automatable. And so that’s how we need to be thinking so that as we um as we implement AIdriven development we not only equip uh the work that softwares software engineers are doing with it but we use it to take smaller bits of the work that is repeatable in nature away from the software engineers and start having that work be automated.
I mean, isn’t that what computers are for in business? Is to automate things. They’re specialized pieces of machinery. And yeah, we’re figuring out new and interesting ways to automate more work. That’s what it’s all about.
And so, you can see in the background, I just I generated some more changes and things just start happening. Okay. So uh uh I’m going to I actually already talked about this slide about uh quantifying how much software you have using using function points to measure the size of the software. And with AI tools now you can you can automate the function point counting. It’s not it’s not a lines of code count.
It’s not a user story count. It’s not a how much have we changed count. It’s at this point right now. What is the economic size of my software system? Okay, let me come back to this concept uh to this question we have.
Is this build of the software good? Is this build of the software good? Um and and so in order to do that we have to define our definition of good. Every aspect of good needs to be mapped out. We need to figure out a consistent way to measure that aspect of good.
We need to find tools that can automatically analyze that aspect of good. Uh we need to where something might be a paragraph or a thousand words, we need to transfer that into a picture. A picture is still worth a thousand words. And these AI tools, boy, do they love to generate text. And I do not like text.
I like pictures. I like tables. It’s the same thing with a with a book. If you’re reading a non-fiction book, do you want to read literally nothing but paragraphs or do you need things that communicate more graphically concepts besides just reading a full sentence structure? Same thing.
Okay. Uh now, why is this even more important to to know if the software is good? because we have more and more pull requests coming through and we need to automate the large part of the answer to that question. This change in this pull request is the state of my codebase still good and what portion of it do I need to review with my own eyeballs? I want to take the portion that I need to review with my eyeballs and I want to shrink it, shrink it, shrink it, shrink it, shrink it because I’ve automated more and more points. and and and when uh when we do this now we can enable more and more people to contribute things. Um I’ll even now I’m not talking about forget regulated industries.
There are certain areas in regulated industries where you have to tightly control the number of people that even have readonly access to the source code. I get that. But there’s all kinds of software that is not sensitive where where you can you you can let more and more people have access to the actual Git repository. And when you give more people access to the Git repository, you and equip them with an AI tool, you get a much more powerful analyst, you much more powerful test designer. But in order to but they don’t have software engineering workstations.
So we have to equip them with environments to use AI tools on the software to analyze something to research something to experiment with something to try something uh to plan something so that they can give a better request to the software engineering team. So, I’m just going to play this video. And what we have here is and this this happens to uh this happens to be clawed terminal in a remote T-M session running inside of a Docker container managed by an Argo workflow that is operating inside a Kubernetes cluster that is running on my Docker desktop installation on my computer. And then I took a video of it. Okay.
And it’s actually running six of these side by side. And so what we have and this is just a test. You can see that the output is very similar. Just so we making sure that we do in parallel. We c uh we took we took different work items you know 7085 7086 7084 and we and we dispatched them to uh to remote environments and and just like I showed you a URL in a browser that was looking dynamically at a terminal session running IBM Bob.
It doesn’t matter what CLI AI coding tool you’re using. You can do a tunnel with a T-Max terminal session for any of them. So it doesn’t matter which coding tool you’re using. But what this does is create an environment, create a prompting environment where anyone can put a prompt to the software. And it’s an environment that can actually compile the code, can actually run the tests, can actually evaluate things.
And a a big problem with involving more people to do anything in a software project is giving them an appropriate environment. And now you can configure, you know, code spaces on GitHub. That’s that’s one that’s one aspect to get here. But but that’s that it is pretty expensive and oftent times we need a lot more flexibility than that and we need to be on demand. And so you know we have an AI coding tool.
Well, yes, we can prompt it. But what’s even what’s even more useful is if I can ask for a field to be, you know, from first name, last name to last name, comma, first name and then it can actually in that environment run the app, give me a tunnel URL so I can so that analyst can see the actual running app and this is just the homepage of the the actual running app and what changed. If I’m changing some text on the homepage or if I’m experimenting with something on a particular page, then what do we do in my development environment locally? I run the app and I look at it and I navigate to that screen and I press a button and I see what happened. But in the age of agile, guess what?
We had tons of contention for the test environment. In the age of DevOps, we learned that well ephemeral test environments might be useful. Maybe we need to spin up some on demand test environments. But that was just spinning up one or two. But on a development environment, every software engineer has a personal development environment.
We can spin up the app. Well, with these with these dynamic uh containerized solutions, which are pretty darn easy. I mean, you put you put these tools together, Kubernetes and Argo and maybe even Codefresh uh from Octopus Deploy and then just add on the AI the AI tools and now any member of your team can have an ondemand containerized environment that not only can use an AI tool to look at something or tweak something, but also run the app right there and look at it to see if you like it. And so you don’t have to go through a pull request cycle just to do that. You just need an environment.
Or what about me? I yes I have my desktop environment but what if I want to run multiple experiments? I can I can spin up multiple isolated environments for myself. Okay. So you need to figure out a solution for your team for ondemand development environments.
In the age of AIdriven development, we we expand our ability to do more in parallel, but we also expand the number of stakeholders that are able to contribute something if we equip them better. if we equip them with some environment to run the run the software and just poke around to do something that that that particular role can contribute. Okay. All right. Um so so under the covers if if anyone’s interested I can give a deep technical demo at another time. Just give us give us a call.
I I mean it’s not it’s not typically hard. You take a few products, you stitch them together and you implement it. And this is and this is part of this is part of the AI software factory pattern which is on demand development environments. Um but but you basically have multiple containers that can do your actual software development life cycle. Not shortcutting anything in your software development life cycle, but doing the actual software development life cycle. and and you want these environments to be to run whatever your favorite tool is.
There’s I mean there’s more coming. There’s all kinds of AI tools out there. You’ve used probably some of the ones on this screen. Grock, Cloud Code, C-Pilot, Bob Shell, and and uh from IBM, and you probably haven’t used some of them. Uh Codeex, Cursor, which is now in SpaceX, but and there’s more and more coming.
Uh there uh Jet Brains Air is in preview. That’s another one. and Jet Brains when they put out a development tool, you know, it’s going to be good. All right, so there’s so many out there uh but you need to be able to equip uh everyone with dynamic ondemand development environment. The bottom line here is that AI, yes, it’s a a significant shift in what we do in our practice in software, but fundamental engineering principles stay absolutely essential. uh clarity, quality, stability, uh architectural thinking, systematic validation with every change. And success in this era of software development and software delivery requires understanding both the capabilities and the limitations of the AI tools while maintaining rigorous engineering discipline.
Rigorous engineering discipline. And so even in the age of AIdriven development, engineering discipline still wins. And that’s the training for today. Um we’re helping more and more clients implement AIdriven development, implement more um more AI tools in a way that actually increases their pace of high quality software delivery. So give us a call if we can advise you, push you in the right direction, give you some training, do a full AI uh implementation, AIdriven development transformation for your team, or if you have an existing project that just needs a little bit of help.
Um, yeah, give me a call. I’d love to talk to you. We do complimentary AI strategy consulting. So if you want to get on with a working session for an hour and just talk through some things, there’s no charge for that. We do that all the time.
So, so yeah, with with your team uh pull me in uh easy hour of just talking about stuff if I can be helpful. So, thank you so much and uh and and hope this was helpful and and please submit your questions uh on the website using our contact us form or send me an email and um and we love empowering your team to deliver worldclass software. So, so um thanks so much. Thanks so much for all your time and uh have a great day. I will continue for those of you who are posting questions now that the the training is over.
I will continue just to answer questions um because I know we have some in there. And uh let me for those of you for those of you who can stay um let me just look at the time. Uh okay. So let’s look at the first question. Um does Claude do things auto can claw do things automatically?
Yeah. So that’s a good question. Any of these AI tools are just AI tools. Architecturally, you put them in an environment where you have them do something automatically or you spin up an ondemand development environment and just be available for somebody to do some prompting. But that’s a good question.
And I think the other questions were done in line. Yeah, they were. Okay, fantastic. So that that’s the questions unless some come in. Thank you so much.
I really enjoyed doing these and thanks for those who submitted questions during the presentation. So I was just able to kind of work it in as I was explaining all the concepts. I love that instead of leaving all the questions to the end. So thanks so much. Uh schedule an AI strategy session with me.
Uh pull me into a team discussion. Um you know anybody anybody want an hour of uh just kind of advice or consulting just to pow through some of these things? Let me know. be happy to do it. And have a uh have a wonderful day. Take care.