When I visited Chicago last, John Bracken and Brian Fitzpatrick aka Fitz from Google organized a very interesting meeting with people from The MacArthur Foundation, Google and various communities including some folks involved in government.
During the meeting, I talked a lot about my thoughts on innovation in the context of newer software development practices and frameworks like agile development and Ruby on Rails. As Reid Hoffman often says, if you're not embarrassed by your the first release of your product, you've released too late." The release early, release often ethos of linux combined with the amount of actual "real work" you can do in one week with Ruby on Rails and other languages and frameworks totally changes the game for early stage consumer Internet investing.
Generally speaking, it's probably cheaper and faster and more effective to make a prototype than to make presentation deck. It's also probably easier to test something on real users than to do lots of marketing and guessing. My recommendation to just about anyone with an idea is to just build the thing, iterate until you have some user traction, then pitch angel investors based on that traction. This is very much in line with the old IETF motto of "rough consensus, running code."
In agile development, you concentrate on doing short iterations with input from your users constantly feeding back into the next iterations.
The "opposite" of agile development is a long process of deciding what to do, anticipating the problems, writing an RFP, work with a contractor until the project is completed, debug it, and then maintain the thing.
The problem is, in the real world, things change and by the time you're done, you're often pretty far off the mark and usually the first version isn't right anyway - so you end up making something 2 years late and a hundred features off target. With agile development, you test, evolve and stay in tune with your users and let them guide you. You can also test and refactor more easily because each "story" or feature is smaller, tested and easy to isolate and remove/change. (Or should be.)
It was very interesting to me that the government folks perked up when we got into this discussion. I remembered a comment by someone at an conference (sorry, I can't remember who said this). The idea was that in big software and in government policy, it was easier to add features (lobby for things to be added into a law) than to remove features. Everyone has their favorite feature that needs to be added. There was very little incentive to remove features and complexity once it was in the law or the code. You end up with things like Windows, some modern cell phones and many of our government policies, turning into bloatware that's huge, too compliated for normal people to understand which doesn't really even do well what it was originally intended to do. I think that keeping units small, proper test suites (accountability at the object level), and agile development can help mitigate some of the causes of bloatware that loses touch with why it exists in the first place and ends up sucking almost all management energy into process.
Also, the idea of floating government policy and iterating rather than taking ALL of the inputs before starting some humongous project also probably makes sense if you have the right kind of structure and discipline. I think there are a lot of things that agile developers have figured out that make sense to look at when thinking about policy and other work.
1 - Extreme programming. Work in pairs on the same screen so that you're checking each other and you're learning each other's productivity and other tricks. Swap partners every iteration. This is a very good knowledge sharing technique.
2 - Test suites. Assume everything will fail. Test what happens when what you have built fails and how it affects the other objects around it. Make sure you will always know when something fails and why. Build system to be robust against human, network, financial, computer failure of the object and build backup systems. Test suites also help you figure out what breaks when you make changes to the system and helps you later when you want to change, remove or refactor stuff.
3 - Small. Keep the teams small, break big problems into small problems. Break the small problems into "stories" or short tasks that a very small group of people can do.
4 - No proposals, specs, RFP's. Use a tracking system like Pivotal Tracker for tracking the tasks, but don't do huge project sheets or try to decide everything before you get started. It's more important for each of the small groups to share their local context and that each small part works correctly and doesn't screw up the stuff around it.
It may seem counter-intuitive, but I think that having a lot of small groups focused on being robust and agile and relatively independent makes it easier for the higher level decisions to be made and retain focus on the mission. Micromanaging is huge and inefficient. Each small group provides inputs to the system and feedback from the "users". Unbundled and small groups makes the whole system much more flexible and "agile" and changes can be made quickly without breaking things and allows focus on context instead of structure.
A lot of my thinking in this area has come from watching Jay Dvivedi and his team at Shinsei Bank and also working a a little with the Pivotal Labs folks. Having written this rambling blog post, I'd still like to say that I'm still rather new to the whole world of agile development, but I think there are a large number of practices that are being developed that can be applied to many other fields including but not limited to government policy development.