1.JPG)
Guessing is not a strategy: How to build decision velocity with AI and real-time data
June 10, 2026 | Live Webinar
ATDD and TDD
| A question that we are often asked is: “What is the difference between Acceptance Test Driven Development (ATDD) and Test Driven Development (TDD)?” These two activities are related by name but otherwise seem to have little to do with each other.
This is a human-oriented interaction that focuses on the customer, identifying their needs. These needs are specified using the external, public interfaces of the system.
All requirements coming from all of these different customers must be addressed, identified and expressed through the ATDD process. For example:
And let us not forget the the developers are customers too, who else do we build tracers and loggers for? This is an obvious, publicly visible facet of the developer’s work. But when do we need these facilities? When we try to fix bugs. When we want to understand how the system works. When we work on the system for any reason. |
Posted on: February 12, 2021 02:21 AM
| Permalink |
Comments (1)
The Importance of Test Failure
| The typical process of Test-Driven Development goes something like this:
There are variations (we’ll suggest a few in another blog), but this is fairly representative. We create process steps like this to guide us, but also to create agreement across the team about what we’re going to do and when we’re going to do it. Sometimes, however, it seems unnecessary to follow every step every time, rigidly, when we’re doing something that appears to be simple or trivial.
So: test failure validates that the test is meaningful and unique, and it also confirms that the code we’re about to write is useful and necessary. For such a simple thing, it provides an awful lot of value.
In TDD, success is not getting to green. Success is the transition from red to green.
[1] And we apologize for the fact that you are now conscious about your breathing. It’ll pass. |
Posted on: February 12, 2021 02:09 AM
| Permalink |
Comments (0)
Mock Objects, Part 1
| Narrow Specifications
If the behavior we are testing/specifying has dependencies on any of these “other” things then we must take control of them in the test so that the only thing we are focusing on is the one thing we are not controlling. Mocks are a big part of solving this problem.
This, of course is a crash test dummy. This is a good analogy for a mock object for two reasons.
+CheckFieldBoundary() is what we want to specify/test. If the GPS reports our location is too close to leaving the planting area, then the BoundaryAlarm object should call ActivateDashLight() on the DashLight interface.
Note that the mocks have additional methods added to them that are not defined in the interfaces they mock:
These extra methods for conditioning and inspecting the mocks will not be visible to the BoundaryAlarm object because the mock instances will be implicitly up-cast when they are passed into its constructor (assuming a strongly-typed language). This is is essentially encapsulation by casting.
Adding more capability to this mock later, or even creating another mock for the purpose of specifying the deactivation of the light, will not be hard to do. What is hard is keeping track of capabilities we may build in anticipation of a need, when we do not know if and when that need will arrive. Also, if this “just in case” capability is actually wrong or non-functional, we can easily fail to notice this or lose track of it.
|
Posted on: February 12, 2021 02:02 AM
| Permalink |
Comments (0)
Mock Objects, Part 2
| Techniques
Perhaps the DashLight implementation is part of our responsibility, and so we’ve opted to create a separate interface which makes mocking easy. The GPS team might not have (note the GPS object above is a concrete implementation which we are directly dependent upon). To test our object (BoundaryAlarm) we must bring the GPS under the control of the test.
Here we have simply sub-classed the “real” GPS class. As was the case when GPS was an interface, our mock will up-cast and appear (to BoundaryAlarm) to be the real thing. The extra method SetCurrentLocation() will be available to the test, for conditioning purposes, because the test will hold the reference to the mock in a down-cast to its actual type.
Obviously if we could change the GPS class, we’d opt to create that separate interface and eliminate all of these problems. But what if we cannot?
GPSWrapper is our class, which we created just for this purpose, so we could mock it. Clearly we would have to change BoundaryAlarm in this case, as it no longer directly depends on GPS but rather on the GPSWrapper, and these two are not interchangeable. But, if we can change BoundaryAlarm and cannot change GPS, then this technique is appropriate.
This eliminates the inheritance coupling between the mock and the implementation entirely, and thus we create no instance of the actual wrapper implementation (GPSWrapperImp) at test time. Our tests will run fast, be completely repeatable, and will not require the actual GPS system to run. These are all techniques for creating mocks by hand. Another approach is to use a mocking framework, a tool that creates these mocks for you. We’ll examine an example of such a tool a bit later on, and also discuss what we like and dislike about the use of such tools. In any case, whether you automate you mocks or write them by hand, every developer should know how to handcraft them, how they work, and what they do.
Part 1: https://www.projectmanagement.com/blog-post/68267/Mock-Objects--Part-1 Part 2: https://www.projectmanagement.com/blog-post/68254/Mock-Objects--Part-2 Part 3: https://www.projectmanagement.com/blog-post/68253/Mock-Objects--Part-3 |
Posted on: February 12, 2021 01:51 AM
| Permalink |
Comments (0)
Mock Objects, Part 3
Dependency InjectionImagine that the example use previously was implemented differently: public class BoundaryAlarm { private GPS myGPS; private DashLight myDashLight; public BoundaryAlarm(){ myGPS = new GPSImpl(); myDashLight = new DashLightImpl(); } public void CheckFieldLocation(){ // Implementation unchanged } } The difference is that the implementations of the GPS and DashLight dependencies (namely GPSImpl and DashLightImpl) are created in the constructor of BoundaryAlarm directly. No matter what technique we use to create mock implementations of these dependencies, it does not matter because the mocks will never be used. Remember, we are forced to test everything which is in scope but which is not under the control of the test. If we write tests for BoundaryAlarm in its current state, we will also be testing the objects (and the hardware) it depends upon. We will have a test that can fail for many different reasons, and thus when it fails we will have to investigate to find out which of those reasons is the culprit. This will waste our time and will drastically reduce the value of the testing effort. We need, somehow, to inject the dependencies into the class we’re testing, so that at test time we can inject the mocks instead, putting the test in control of the dependencies (leaving only BoundaryAlarm’s implementation as the thing being tested). Let’s note that the example above really contains a bad bit of design. The BoundaryAlarm class is really two different things: it is the consumer (Client) of the GPS and DashLight services, and the creator of them as well. Any given entity in a system should have one of these relationships, not both: the client of a service couples to the way the service is used, but not necessarily its concrete type, whereas the creator of an object always couples to its concrete type. Here we feel the pain of this bad design decision in the difficultly we’re having in trying to test just what we wish to specify, and no more. Here again this is a question of technique, and here again we could probably come up with any number of ways of solving this problem. We need to know more than one way because the context of our problems will be different at times, and a given technique that worked will in one situation may be totally inappropriate in another. We’ll look at four different ones here. 1. Direct Injection This is basically what we did initially; we allowed the dependency to be handed in via the constructor (and, btw, from this point forward we’ll just use a single dependency for brevity -- GPS). We could, alternately, have provided a setter: public class BoundaryAlarm { private GPS myGPS; public BoundaryAlarm(){ myGPS = new GPSImpl(); } public void setGPS(GPS aGPS) { myGPS = aGPS; } public void CheckFieldLocation(){ // Implementation unchanged } } This makes it possible for the test to inject the mock, but note that the setter does not remove the new GPSImpl() offending code from the BoundaryAlarm. If we go completely back to the previous approach: public class BoundaryAlarm { private GPS myGPS; public BoundaryAlarm(GPS aGPS){ myGPS = aGPS; } public void CheckFieldLocation(){ // Implementation unchanged } } Then BoundaryAlarm is a client of the GPS service, but is not also the creator of it. The problem, of course, is that any entity (not just the test) can hand in any implementation of the GPS service, and this could represent a security problem in some domains. Some unscrupulous individual could create an implementation that did something unwanted or illegal, and we’ve left ourselves open. It’s hard to imaging such an implementation of a GPS unit, so here direct injection might be just fine… but imagine a banking application with dependencies on a funds transfer subsystem, and the possibilities become alarming. So, another way to go is to use a design pattern: Service Locator 2. The Service Locator Pattern The service locator approach basically says that it’s better not to have a client object create its own service object(s), but rather to delegate this creation to a factory or registry object. The implementation of such a factory could vary widely based on the nature of the service object(s) being created, whether there is variation in their types or construction, whether there are multiple clients with different application contexts, etc… In our case, there is only one client and only one service version, and so we’d implement the locator simply: public class BoundaryAlarm { private GPS myGPS; public BoundaryAlarm(){ myGPS = GPSLocator.Getinstance().GetGPS(); } public void CheckFieldLocation(){ // Implementation unchanged } } class GPSLocator { private static GPSLocator _instance = new GPSLocator(); private GPS theGPS;
private GPSLocator(){ theGPS = new GPS(); } public static GPSLocator Getinstance() { return _instance; } public GPS GetGPS() { return theGPS; } // For testing public void setGPS(GPS aGPS) { theGPS = aGPS; } public void resetGPS() { theGPS = new GPS(); } } Note that the service locator () is a singleton [1]. This is important because we need to ensure that the instance of the locator being used is the same one that the test will interact with. Also note the “for testing” methods that allow the test to change the implementation that the locator returns to something beneficial for testing (a mock, in this case) and then reset the locator to work in its normal way when the test concludes. Of course, this could represent a similar security threat as with direct dependency injection, and so most likely these additional methods would be removed when the code was shipped. The advantage here is that client objects tend to grow in number whereas locators and other forms of factory objects are much less likely to do so, and so we’ve reduced the amount of maintenance we have to do. 3 – Dependency Injection Frameworks As with the creation of mocks, injecting them can also be done with automation. A dependency injection framework (or DIF) can essentially provide a service locator for you. There are many different tools for doing this, and as we do not want to focus overmuch on tools (TDD is not about tools per se) we will simply give a single example. We are not advocating for this particular tool, it’s just one we’ve seen used fairly frequently. It is called Structure Map [2], and it happens to be a .Net tool. There are plenty of such tools for Java and other languages/frameworks. First, we change the BoundaryAlarm code: public class BoundaryAlarm { private GPS myGPS; public BoundaryAlarm(){ myGPS = ObjectFactory.GetInstance(); } public void CheckFieldLocation(){ // Implementation unchanged } } ObjectFactory is a class that is provided by Structure Map. For it to work, we need to bind to a resource that will map a type referenced to a real type. Structure Map can actually map to many different kinds of resources, but we’ll choose XML here as it makes for an easy-to-understand example. // In StructureMap.config
PluginType="GPS" PluggedType="GPSImpl" ConnectionString="...." />
All this does in ensure that every time the type “GPS” is specified, ObjectFactory will return an instance of GPSImpl. Now, in the test we do this: [TestMethod] public void testBoundaryAlarm() { GPSMock mock = new GPSMock(); ObjectFactory.InjectStub(typeof(GPS), mock); BoundaryAlarm testAlarm = new BoundaryAlarm(); //The body of the test, then ObjectFactory.ResetDefault(); } Again, this is just an example using this particular tool. The advantage here over writing your own service locators is that these tools typically have a way of disabling the injection (disabling the InjectStub() method in this case) before the code is shipped, which further reduces code maintenance while not leaving the “door open” for miscreants in the future. 4 – Endo Testing [3] Our first three techniques all centered upon the idea of “building the dependency elsewhere”. In direct dependency injection, it is built by the client/test. When using a service locator, it is built by the locator. In using a DIF, the framework tool creates it. Endo testing is a way to avoid creating this separation while still allowing the dependency to be injected. Techniques are good to know about, but beware the overuse of any code tricks you happen to know. Remember that good separation is probably a good idea anyway, and just because you know how to do something does not mean you should. To start, we change BoundaryAlarm in a different way: public class BoundaryAlarm { private GPS myGPS; public BoundaryAlarm(){ myGPS = MakeGPS(); } public void CheckFieldLocation(){ // Implementation unchanged } protected virtual GPS MakeGPS() { return new GPSImpl(); } } We have not gone to the extent of creating a new entity to manage the instantiation of the GPS dependency, we’ve simple done it in its own method. This would be a very simple refactor of the original code, something most IDE’s would do for you as part of their built-in refactoring tool suite. However, note that the method is both protected and virtual (marked that way in a language like C#, or that way by default in a language like Java). That’s the trick: it allows the test of BoundaryAlarm to actually create a subclass to be tested. Ideally, this will be a private, inner class of the test: [TestClass] public class BoundaryAlarmTest private GPSMock mock; [SetUp] public void initializeTest() { Mock = new GPSMock(); } public void testBoundaryAlarmRespondsToGPS() { BoundaryAlarm testAlarm = new TestableBoundaryAlarm(); //The body of the test, conditioning the mock //as needed } private class TestableBoundaryAlarm : BoundaryAlarm { protected override MakeGPS() { return mock; } } } This trick can be used to solve other problems involving dependencies. For example, we often develop code that couples directly to system/frameworks entities such as the date/time API, a network socket, the system console, random number generation, and the like. We can wrap access to simple entities in local methods, and then override those methods in the test. Let’s take the system console. If you’re writing directly to it, it’s very difficult for the test to see what you’ve written, and specify what it should be. You could create a mock of the class that represents the console, and perhaps you would. But, if it was a simple issue and making a mock seemed like overkill, you could wrap the console in a local method. public class MessageSender { public void SendMessage() { Output(“Ground control to Major Tom.”); } protected virtual void Output(String message) { Console.WriteLine(message); } } Now, in our test, we simply subclass MessageSender, override the Output method to simply write the message into a local String, or keep a log of all writes, or whatever is most convenient to the test. You could do the same with a network connection, or an http request, or… whatever. Conclusions All systems are built using dependencies between entities and yet these entities must be tested in isolation from one another, in order to create narrow, precise specifications of their behaviors. The ability to create and inject mocks of other entities when testing a given entity that is dependent on them is crucial in TDD. The more techniques we learn, and the more we innovate as technology changes, the more efficient we can be in breaking these dependencies. One thing we will want to examine is the role of Design Patterns in all of this. If a pattern is (as we believe) a collection of best practices for solving problems that recur, surely the pattern should include best practices for testing it. Once we figure out how to test, say, the Chain of Responsibility (where the mock object goes, what it does, etc…) we see no reason to ever have to figure it out again. So, stay tuned for that discussion! [1] Unfamiliar with the Singleton Pattern? See: https://www.pmi.org/disciplined-agile/the-design-patterns-repository/the-singleton-pattern [2] And you can get it here: http://docs.structuremap.net/ [3] This technique was originally suggested by Alex Chaffee and Bill Pietri at IBM’s Developerworks: see http://www.ibm.com/developerworks/library/j-mocktest/index.html |
Posted on: February 11, 2021 08:49 AM
| Permalink |
Comments (0)