Want to get it done? Lessons I learned from applying agile to writing

We all have something we want to finish. Some of us haven't started yet. Some have. There are lots of different ways to go about tackling the job. This is a story about one way that applies in many situations.

Chances are, if you've seen me in real life, you've seen me writing. So you might be surprised to learn how I finally finished the novel I've been working on for the past four years. This isn't just a story for people who are trying to finish a novel. It's for anyone who is struggling to finish any project, and for people who feel overwhelmed by the enormity of an ambitious goal. It can be done, and inspiration might come from where you least expect it.

The Tedium of Creativity

At the beginning comes the euphoria of an idea, the thrill of being so far away from the possibility of achieving it that the only thing you really care about at that moment is how brilliant this idea is. Reality then begins to set in. Maybe your idea isn't brilliant. Maybe it's extremely complex and you have no idea how to make it feasible. It is at this point that imagination needs to get applied. Ideas are not difficult to produce. Taking ideas out of the realm of abstraction and making them real is so much harder. That is what I call the tedium of creativity. The grind of ten thousand little steps between that initial euphoria and the even greater thrill of manifesting that plan. This process can wear you down.

For me, this particular euphoric idea arose four years ago, when I went to Italy to get my great-grandparents' birth certificates. Everyone at the front desk of the hotel, it turns out, was related to me and asked what I knew about our family history. In classic form, they rolled their eyes and dramatically lifted and dropped their left eyebrows when I said I had no idea. They sent me off to a castle to learn. And that, my friends, was when my life changed.

At the medieval castle, I became lost in a world of wonder. By the time I left, I had a completely different sense not only of myself, but of the world. I was shaking from excitement. But what should I do with this knowledge and energy? Turns out, of course, that I didn't know anything yet, but I was deep in a rabbit hole and there was no turning back. And so began four years of research and writing during every available second that wasn't occupied by Science House, the company for which I am the EVP for Business Development. Our clients directly benefit from everything I learned about the greatest minds and generals of the Renaissance, even though most of them had no idea I was writing this book. But what I didn't expect was how my work at Science House would enable me to finish this project.

How it Went Wrong

My father is a writer. He got me started when I was a very small child. He was, at the time, a student of Gordon Lish, who is famous for his savage editing of some very famous writers, including Harold Brodkey, Raymond Carver, Don DeLillo, Barry Hannah and Joy Williams. My father's ego had a very difficult time handling this environment, and he turned the Lish treatment on his little daughter, for which I am extremely grateful. I didn't know then that this harsh reality was the beginning a lifelong habit of deliberate practice. This requires the practitioner to welcome, not just tolerate, criticism, so weaknesses can be identified and improved.

Starting in childhood, I wrote a manuscript for a novel just about every year. By the time I was an adolescent, I would send these manuscripts to industry professionals, not for the sake of publication, but to solicit criticism. Then I would start over with a new story, focusing on the weaknesses from the last. Sure, I would cry in the shower for a few days first, convinced that I would never write again, that it was a form of masochism I needed to abandon. But I never abandoned it and I never will. I love writing, even though it is often lonely and strange to carry a dynamic virtual world in my head at all times while I'm living my real life.

After I left the castle, I knew I finally had the greatest story to tell. But for some reason, instead of writing a novel, I decided to write a history book. And it was so dull. Not the process. I loved doing the research and writing the book. I wrote the entire book before I realized that it sucked. A year of my life, vaporized. Or was it?

Alexa, Can You Write My Book?

Around the time I realized that I needed to start over and make it a novel, Amazon invited me to deliver keynotes to three hundred of the company's top technologists. I should mention here that I am a futurist, and my primary work at Science House is advising leadership teams to help them create the future they can imagine. Many of these teams are either executing or involved with massive software projects. My work is extremely pragmatic, and ranges from high level cultural issues such as leadership development to helping organizations fix meetings, where time and resources tend to get squandered.

I started doing a lot of research for this engagement. Information and ideas about voice activated technology mixed with my research on the Renaissance. Next thing you know, I was done with draft two, and it was two books in one. One was a science fiction novel about the future of a society completely permeated by a layer of invisible surveillance, and the other was a perfectly factual account of the story from five centuries ago that I actually wanted to write, in a completely different voice, as if two different people had written it.

When I came back from the west coast, I got serious about my third draft. It turned out to be the worst of the bunch. After editing out the futuristic storyline, I was finally alone with the heart of the story. Boy, did I butcher it.

How Software Architecture Saved The Day

In 2006, when I first started my own boutique strategic consultancy to develop the Imagination Age, IBM became my first client. I had no idea how lucky I was to meet Grady Booch and get introduced to the topic of software architecture from the master himself. He will laugh at me for saying this, but early on, we attended a conference together and it was as if Zeus had made an appearance. The significance of Grady's contributions to the field can't be overstated. I was already interested in software architecture, but Grady ignited that passion and has remained a visionary guide in all the years since. I spend so much of my time working with software architects, and this path would not have been possible without Grady helping me understand the art of the possible.

So here I was, faced with the decision to give up on my book or start over again, when I found myself working on the development of a team that is beset with software architecture issues. I will write more about this topic in 2018, but for the sake of this post let's just say that many companies believe they have a much stronger understanding of the term "agile architecture" than they really do, and the assumptions they make about the practice cost a lot of time and light a match to funds while burning people out. Given the pervasiveness of this issue across industries, business has been brisk at Science House. It is easy to identify this problem in clients, because it is so evident, but it wasn't easy for me to see that I was having the exact same problem with my book.

I was coding without a plan. I thought I had a plan, which is exactly how I ended up with three bad drafts. I was willing to scrap it and start over, keeping the core of the story each time while getting closer and closer to what is should be, instead of what it became when I plowed ahead.

Words are how the architecture of a story is made visible, just as code is the way the architecture of software is made useful.

"Architect" is a term that gets thrown around these days. When I stopped writing and started grappling with the project as an architecture challenge, everything changed. The connections underlying the entire story began to take shape in a new way. I understood it at a 30,000 foot level, and also at the granular level.

Of all the powers we are losing as a society, the most dangerous loss is that of thinking. Taking the time to stop and think before plowing ahead. We are all so busy all the time, so distracted, so desperate to finish the thing we started even if we're headed down the wrong path. No senior executive wants to stop and explain to the board of directors how it all went wrong, and what's being done to fix it. No team leader wants to explain how complex a project really is to senior executives. The people doing the work often have a much clearer picture of reality than the people who planned it. Our relationship with the concept of failure is riddled with ego issues, but we should expect that along any ambitious path, we are going to see a better way to design the outcome, and we should pursue it.

With a novel, the writer plays all of those roles. It's much easier to change course when only one person is affected, but it's also very painful for one person to admit that the whole process needs to begin again, without illusions. The same emotional issues apply. The same time constraints press hover like an hourglass losing more sand every second.

I'm not going to say that writing the last draft of my book was easy. But it was like being in a field with open space instead of a room filled with clutter. I was convinced that all of the clutter was so important that I couldn't let it go. In the earlier drafts, I was more focused on myself as a writer and less focused on my future readers, the same way many large companies lose sight of their customers while they are lost in the coding process. We are architects, and we need to put the hard work into the design so that the execution can be achieved to the benefit of those for whom we are making our creations, after all.

‍