Automating Test Creation

Eric Holscher just posted a very nice article titled Automating tests in Django. The post goes through how to create integration tests for your Django applications in an automated way through the use of a Middleware that logs the test creation output to a file. It’s a creative approach and certainly very interesting. One additional benefit is that Eric created a screencast to go along with the post that is excellently done.

There is one thing about this approach to testing that doesn’t quite sit right with me and that’s that it seems like the testing process is backwards. If you’re creating tests based on what you have how are you possibly going to cover what’s specified but not implemented properly? It’s the same reason I’m not a fan of doctests. I think they encourage the wrong behavior, especially when often the output your matching to is so complex that the tendency is to just copy and paste from live results. I recognize that a lot of people don’t feel the same way, and perhaps I just need to give the idea more time to sink in.

I really appreciate all of the screencasts that are starting to show up within the Django community. I think it’s a vehicle that a lot of people enjoy and learn well from. I know that I’m certainly looking foward to more screencasts from Eric.

Discussion

24 Jul 08

Ferry Boender says:

I believe Emil is right. It’s commonly called ‘regression testing’ (http://en.wikipedia.org/wiki/Regression_testing), and it can be an enormous help when your working on a piece of code with multiple people or when you’re refactoring or fixing bugs, etc. One particular thing where good regression tests come in handy in particular is when porting Python applications from version 2.5 to 2.6 and 3.0, for instance.

24 Jul 08

Emil Stenström says:

I think one of the reasons people like the “copy way” of making tests, is that it protects from breaking things that “worked before”. I can see their point.

Thanks for a good blog.

Empty Thoughts