I’ve been playing around with Graphviz this weekend. I first used it a few years ago with a Perl script that sorta kinda knew Cold Fusion and JavaScript syntax and could output the Graphviz .dot file format, as a means of visualizing all the dependencies between source code files in a project with no compilation phase and no automated tests. It helped me answer a few questions that I had about the code: What should I write tests for first? What should I leave alone, because breaking it breaks a bunch of pages? What pages do I need to test in order to make sure that changes to a deeply-buried chunk of included code didn’t break anything? Having a simple tool that draws graphs of nodes in a fairly clean form can be really useful.
This time, I’m trying to visualize the dependencies among not-yet-written features of my current software project. I have an ERD, so I can use foreign key relationships to determine hard dependencies between tables, but I want to understand functionality in terms of larger functional chunks. Call ’em modules, components, or whatever; I want to know what needs to be built first, and how much of it I really have to build before I can build stuff on top of it and get a minimal system assembled. Ideally I’d like to be able to put together a core set of functionality and then launch, leaving the prioritization of what to do next until later. At that point I should have user feedback, some experience with the low level details of the app, some issues in an issue tracker etc. to guide my decisions.
There’s a Mac port of Graphviz that’s particularly nifty because it watches the input .dot file and updates the already-displayed drawing immediately when that file changes. My first thought was to use the subgraph clustering feature of “dot” to group tables from the ERD into modules, but that didn’t work because dot doesn’t do cluster-to-cluster edges. Instead it connects nodes, meaning tables, which resulted in a really amazingly ugly graph without clustering, and in an endless “Rendering…” message with clustering. I think the problem I gave it to solve (how to lay out the graph while still grouping all of those clusters of nodes in little rectangular boxes) was too complicated, and I don’t feel like gradually building up to that stage to see what the problem was or waiting hours to find out how long it’d actually take. So, I’ll just manually define functional clumps, and map out dependencies, and then try and define for each of them the minimum needed functionality to support the stuff that depends on it.
It would sort of be cool to have an uber-graphing database modeler / class browser / IDE / documentation tool, but so far every product I’ve seen that tries to do that is an unspeakable usability horror that makes you do all sorts of obtrusive stuff to your code and which runs slowly and has lots of bugs and is astonishingly expensive if you want the grown-up version. It’s way past the point of diminishing returns to try and find a tool that will model the database and classes and modules and requirements and keep it all in sync. Better to just get 80% of the way there by hand, make a decision, and then abandon the diagram and move on.
I’m also trying to get to a point where I can create a fairly good effort estimate for each feature in isolation, so that I can start playing the “what has to get cut to meet a given launch date” game. That’s kind of like The Planning Game, except that I have a “big picture” in mind and I’m just trying to take chunks of it and fit them into a short time frame. The Planning Game from XP seems to be more about distrusting the claim of an accurate understanding of the big picture, and just adding functionality that makes sense with each successive iteration.
I disagree with that point of view, but I think I have enough big picture detail figured out that I can avoid painful and tedious rework later. I know what I think the full implementation of each part will look like, so I think I can avoid doing things that will cause problems later.
The next question is whether I should try and create a “big estimate” based on what I think I want to build, given the “big picture” design I have in mind. I think that’s moot at this time; what I need is a definition of a minimal set of useful functionality, and an estimate for that. The estimate for later features can come later. I suppose that with a huge amount of money involved and lots of anxious investors, I’d need to spend more time on accurate estimation and project planning and less time on actual implementation, but that probably also would depend on investor wisdom. In other words, a reasonable idea of the big picture should be enough for smart developers, and a steady succession of new releases that add functionality should be enough for smart investors. Only if the financial bet is a really risky one would anyone need a serious estimate of where things will be in 2 years, and really, that’s almost impossible to predict anyway, and the prediction process itself may decrease the chances of success.
So, it’s time to carve out a chunk of work (using Graphviz to help me carve intelligently), make sure it fits a reasonably short time frame, and build it. Exciting!