RyanHagan.Net

The Mythical Man-Month

I’m often surprised by how many people in the software industry haven’t read Frederick Brooks.  There are few people in our industry that have written truly, seminal books, and Fred Brooks is one of those people.  There are even fewer people that can claim to have postulated a law that is still quoted 35 years later.  Again, Fred Brooks is one of those people.

The Mythical Man-Month is a collection of essays on the topics of software engineering and project management which was published in 1975.  It’s famous for originating Brooks’s Law, “adding manpower to a late software project makes it later”.  Almost everyone that works with software (developing or managing) has heard, and possibly even repeated, Brooks’s Law.  Unfortunately, few people understand why it’s true and what they can do to avoid falling into that manpower trap.

The book is by no means a one-trick pony.  There are dozens of essays in this book that are incredibly valuable to those of us in the software development industry.  In one of the essays in the book, Brooks suggests that the waterfall model of development might not be the best way to work, and proposes something very similar to what many people, today, would recognize as an agile process.  Of course, he also comes down very squarely in the waterfall camp in another essay.  He explores the benefits of “build vs. buy” (the idea of buying commercial, off the shelf (COTS) software in 1975 was very novel), likens software development to prehistoric tar pits, proposes the second-system effect, explores operating software development teams in the same way surgical teams are operated, and many other fascinating topics.  Reading through these essays provides an amazing insight into how projects went wrong in the late 1960’s and early 1970’s, followed by immense frustrations that we’re still dealing with the exact same problems today.  Really, more people need to read this book.

Brooks wrote his most controversial essay, No Silver Bullet, in 1986.  Quite simply, Brooks states that there will be no new developments in the creation of software or management of projects that will provide an order-of-magnitude productivity gain within the next decade.  He proposes that software development has two primary problems, essential complexity and accidental complexity.  The essential stuff basically boils down to solving problems, while the accidental stuff tends to focus around the actual writing of code.  He says that so much of what we do falls into the essential camp, that even reducing the accidental complexity to zero wouldn’t result in the order-of-magnitude improvement that we kept seeing in the hardware space at the time.  Here we are, 25 years later, and I don’t think we’ve still seen those kind of productivity gains.

I can’t recommend the works of Fred Brooks highly enough.  Even if you don’t agree with some of his proposals and conclusions, reading about them is fascinating.  The insights that can be gleaned from his works are nothing short of astonishing, especially when looking at them through the lens of the twenty-first century.

We do performance reviews once a quarter at my current company.  Without exception, I’m the first one done with my reviews and I kind of enjoy the looks I get from my peers when they’re scrambling to get reviews done before the deadline while I’m living stress free.  Well, I have a few secrets that’ll I share with everyone for stress-free, yet still effective, performance reviews.

My first secret is to simply prepare early.  I schedule appointments to prepare for performance reviews well in advance.  Usually I’ll schedule these for the next review period as soon as I finish the current reviews as part of my wrap-up process.  I’ll schedule appointments for requesting peer feedback (this is just an email I send out for each of my directs to the people they worked most closely with during the review period), salary review (to simple make sure everyone’s salaries are where they need to be), transcribing notes from one-on-one’s during the review period, and coordinating with any managers that managed one of my directs during the previous review period.  All this is done before I actually begin writing up the reviews.

My second secret is to treat the entire quarter as the review period.  I keep a notebook for each of my directs.  In that notebook, I record every one-on-one, I write down every success that the direct has, and I write down every time I hear my directs express concern or frustration over a particular challenge which often end up turning into goals.  All this information ends up in the review at the end of the quarter in one form or another.

My third secret is to leverage others in the organization as much as possible.  I mentioned above that I collect peer feedback, which is great for coming up with strengths, successes, and areas of improvement feedback on the review.  Promise to keep the feedback anonymous and you’ll generally get great ideas to use in your review.  Leverage your product owner (or whomever sets priorities in your organization) for goals.  Turn the highest priority into a goal that fits each team member.  Lastly, of course, leverage your directs.  Ask them to come up with some personal development goals that you can help them achieve over the next three months.

Putting all these together makes for reviews that practically write themselves, but remain very relevant for your directs.  It takes me thirty minutes to deliver the reviews, and I just do them during the regular, weekly one-on-one that we have scheduled.

As I mentioned in a previous blog post, we decided to try an experiment with one of our larger software development teams.  The idea came from the fact that we spend a lot of time telling our employees that we encourage them to work on 20% projects, but very few people actually make the time for them.  They have a lot of day-to-day responsibilities, often with deadlines attached, and spending 20% of their week working on an unrelated project without a deadline just doesn’t happen as often as any of us would like.

Therefore, we cleared all our calendars for a week and told our developers to go nuts.  Work on any project you want to work on regardless of how closely it matches your current job duties, or even what we do as a business.  In other words, there are zero restrictions on what you can work on.

The format of the week was pretty simple.  If anyone needed any special equipment (2×4s, ping-pong balls, an arduino, etc.) we made sure to purchase those in advance so they were here on Monday morning when we got started.  Everyone worked on their projects throughout the week as they would normally.  On Friday, we ordered lunch for the team while we prepared the room for the demos.  We invited anyone from the company down to watch the demos, which made for a really great audience.  Each demo was time boxed to five minutes and once the demos were done, we sent the team home for an early start to the weekend.

The week was an amazing success.  People worked alone, people worked in teams, people worked with peers that they rarely get a chance to work with.  We even convinced one of our designers to sit down here for the week to help us with projects that need art assets.  Some of the projects that people worked on were a physical continuous integration status display for our projects, a massive refactoring of our search project (45,000 lines of code REMOVED), an iPhone app to manage passwords for our email customers, an Android video game, and a “Minority Report”-esque physical computer control using a glove, infrared sensors and a Wii remote.

We saw an amazing amount of passion and creativity during the week.  The amount of autonomy that was given to our employees during the week was something they had never experienced at work.  All of the good will and the excitement that was generated, easily flowed over into the following weeks and resulted in higher quality work (fewer bugs than the average) and higher productivity (as measured by sprint velocity) than the weeks prior to the 20% week.  Furthermore, the employees that participated were much happier than we’ve seen them in a long time.

Obviously, we’ll continue to put these 20% weeks on the schedule.  We’ve decided to do two of them per quarter, one about three weeks after the start of a quarter and the other about three weeks prior to the end of the quarter.  This spaces them out nicely to about once every six weeks throughout the year and is actually about 17% time.  We’ll even start video taping them to share with a wider audience and hopefully spark the same idea with other business units and companies.

As I continue to advocate for the abolishment of vacation and sick time, I hear a lot of business owners and managers balk at the prospect.  So far, the most common argument against goes something like this.  How can I possibly fire someone that abuses a policy that I don’t have?  First, do you really need to fire the person, or are you just avoiding having a difficult conversation with a direct report?  Second, is creating a policy really the only way you can deal with the situation?

I really hate seeing managers work so hard to justify the existence of their current vacation policy, or any other dunderheaded policy, for that matter.  To me, this signifies a giant lack of creativity.  Why do we punish 99.5% of our employees because of the 0.5% that will cause problems?  Creating a new policy in response to the undesirable behavior of a single employee is gross negligence by your managers and your company. All you do is generate resentment in the large portion of your employees that weren’t causing a problem, while avoiding having a difficult conversation with the employees that were.  Is it really worth it?

By way of an example, I worked for a medium sized software development company (about 50 employees) a few years back.  While I was there, one of our employees conducted an interview while wearing a t-shirt with the words “what the f**k are you looking at” printed prominently on the front.  Obviously, this was a problem and very poorly represented the company.  The candidate who was being interviewed was asked to come in for a second round, but told us he wasn’t comfortable continuing the interview process.  The result of all this was the company implemented a “no t-shirt” policy.  Due to one incident by one employee, that company banned t-shirts.  This is NOT the right way to handle this incident.

If you don’t believe me, then listen to someone that is way better at building a highly engaged work force than I.  Marisa Keegan writes for the blog Fistful of Talent, and basically says the same thing.

Where did everyone go?

Stop tracking employee vacation and sick time.  Just stop.

Most companies tend to lump sick days and vacation days together into one chunk of time called Earned Time Off (ETO) or something similar, although there are still a good number that keep sick time separate.  Either way, stop tracking both.  Here’s why.

The current policies encourage employees to come to work sick.  Either they don’t have any sick days left, or they don’t want to lose their vacation time because of the flu they just picked up.  Instead of staying home and resting, employees come to work while sick, which causes quite a few problems.  First, they spread their germs to other employees, which probably causes even more people to miss work.  Second, people that try to work while sick are not productive people.  Third, it takes much longer to recover from their illness.  The productivity hit you take from having sick people at work is considerable.

Another problem with current policies is that they foster resentment.  How would you feel, as a dedicated employee, having to cancel your family, Christmas vacation because you got the flu at the start of fall?  There is zero chance of current policies generating any goodwill whatsoever.  None.

But if I stop tracking time that employees take off, won’t all my employees just stop coming to work?  Abolishing the tracking of vacation time requires both the company and the employees to extend more trust to each other.  Employees need to know they’re not going to be punished for taking time off, and companies need to know that employees aren’t going to abuse the new policy.  If an employee does abuse the system, chances are you have other problems with that employee, and that person probably needs to part ways with the company.

I’m sure we’ve all heard of Google’s 20% time by now.  Amongst developers, it’s a very popular idea, and one that several companies have tried to implement in their own way.  Some companies try weekend long hack-a-thons, some companies try 24 hour “FedEx” days, and some companies simply try to recreate 20% time at their own organizations.  What they all have in common is the goal of increasing motivation and engagement at work.  In today’s world, and the thought industry in general, self-direction and autonomy are key concepts behind “motivation 3.0“.

At Rackspace, we’ve tried all of the above, as well as some others that weren’t mentioned.  Each has had their own pros and cons.  For instance, we ran into the same problem with 20% time that Atlassian seems have to discovered.  Basically, very few people would actually take advantage of the time unless it was specifically reserved by a manager, and when you have to be told to work on 20% time, it really takes away the magic.

So, we’re trying a new experiment.  We’re going to take a small number of teams, and for an entire week we are going to work on projects of our own choosing.  There will be zero boundaries placed on the work that you do during this week, other than you have to demo your project on Friday morning.  We’re going to do this twice a quarter, which will amount to something just a little under full, 20% time.

If projects during this week don’t have to be related to work, what value does the company get from sponsoring them?  I’m glad you asked.  As already mentioned, increasing engagement and motivation are huge factors for trying to get this right.  Allowing developers to have autonomy at work goes a long way towards this goal.  Furthermore, any work that our teams do in development, especially exploratory research, is like sending them to training without the total cost.  The innovations that come out of these types of projects will often, even if it’s indirect, benefit the company.  Also, and probably not wholly unexpected, many of the projects that people pick do end up being directly related to what we do at Rackspace, even if it’s not what that team normally works on from day to day.  Finally, even the anticipation of this week of undirected development time is increasing engagement. The discussions about what projects to work on definitely generates excitement, and some people have already built prototypes of their projects for the week.

Once we’ve finished the week, I’ll report back on how things went.  Stay tuned.

The green line is what we want to see in a continuous improvement scenario. Unfortunately, the red line is much more common.

We switched to Scrum, and now we’re seeing fewer releases, more bugs, and engagement is starting to tank.  I’ve seen this time after time, and I’m here to help.  First of all, it’s likely that you have just as many releases and bugs now as you did before adopting Scrum, but Scrum puts this reality front and center and makes it a lot harder to ignore.

Regardless of your development process, eventually you’ll run into problems for any number of reasons.  It’s very easy to get stuck in bug fixing mode, and not be able to see the lack of progress.  Often I feel like I’m the busiest when I spend all day tracking down and fixing production bugs.  That may be true, but that hardly makes me productive.

First of all, it’s entirely possible that you simply don’t notice how much time you’re wasting on bugs.  It’s all too common not to notice at all until a deadline creeps up on us, or a manager (or customer) asks us where some feature is.  I find it extremely useful to not only track bugs (Trac, BugZilla, Jira, etc.) but also to keep reports and trending data on bug stats like total turn-around time, number of bugs per week, first response time from developers, and so on.  You’ve probably heard this before, but it needs repeating, you can’t fix what you don’t measure.  Putting that report in front of the developers is an easy way to get the attention of developers and help them take a step back to see what the problem is.

Once we can all agree that we’re spending too much time fixing bugs, we need to figure out a way to correct the problem.  I’ve done this enough times that I can pretty quickly give you the answer, but I find that encouraging a team to create or update the process themselves is much more engaging and gets a lot more buy-in than me just telling them what to do.  I still influence the process by agreeing to document the change and asking specific questions like, “how do you define ‘done’ in that step your just described,” or “how does a task move from testing to deployed?”

A really important step of creating or updating a process is to figure out how to appropriately reward or punish team members that don’t follow, or break their part of the process.  Most of the time I suggest something really simple like just tracking the number of times a team member follows versus breaks the process.  I’ve seen teams do things like create a money pot that you have to feed when you break the process, make members that break the process do push ups, and all sorts of other things.  Again, whatever the punishment / reward is, the entire team needs to accept the decision.  If even one person holds out, go back to the drawing board.

As with most things, we’re striving for continuous improvement here.  Give the new process a few weeks and see what works and what doesn’t.  Make changes, but make sure that process is documented and standardized before changes are made.  You want to make progress, but you don’t want the good parts of the process to suffer when new parts are added.  Create a solid base, take a step up, get settled, and take another step.

I’ve been reading a lot about performance reviews recently.  There have been quite a few, prominent publications and respected individuals asserting that performance reviews today actually hurt employee engagement rather than helping.  Certainly, I make no argument that badly executed, annual performance reviews are a bane to any company and should be done away with.  Unfortunately, very few of these authors have suggestions for what to put in place of the performance review, and I feel strongly that “nothing” should not be an option.  Instead of abandoning performance reviews altogether, managers should examine their current practices and discover what isn’t working.

By far, the most common argument against performance reviews is that the goals are rarely relevant by the end of the review period. The problem with this is that the review period is often a year long.  Many companies’ priorities will change several times within that period, and without constant attention to the goals you’ve defined they’ll quickly fall into irrelevance.

There are two, easy remedies for this problem.  One solution is to simply shorten the review period.  I’ve certainly had a lot of success conducting performance reviews once a quarter, which strikes a great balance between the management of shifting priorities and still giving employees enough time to make meaningful progress on a well written goal.  In all honesty, do you believe that you and your directs can create a set of goals that will remain relevant during the course of an entire year?  Unlikely.

The second solution is right out of Performance Management.  That is, performance reviews shouldn’t just happen once a year or once a quarter, they’re a 24/7/365 activity.  Here at Rackspace, with regard to performance reviews, our mantra is “no surprises”.  Through constant communication, management by walking around, and effective one-on-ones, your directs should always know how they’re doing.

Performance reviews, in and of themselves, are not the problem.  Reviews done badly are the problem, and that is simply bad management.  Remember, reviews are not something that you do to an employee, they’re a codification of all the conversations that happen every day.

How important is vision? Let’s start with an exercise. If we’re going to take a trip, would you rather know your destination from the beginning and plan your trip with that in mind, or would you rather have a series of directions presented to you a few at a time until you were told you’ve reached your destination? Start in San Fransisco, get on your motorcycle, and drive north to Portland. Once you’re there, continue north to Vancouver. Next stop, Anchorage. I don’t know about you, but if I’d known my destination was Alaska from the beginning, I would have brought the SUV…or flown.

Software developers often run into a similar problem. Clients tend to give us a list of things to do without ever providing a bigger picture of the end goal. We can only make decisions based on the information that we have, and if there is missing information, we may not make the best decision for your software.

Don’t just take it from me. There are plenty of very well respected authors that stress the importance of vision. Stephen Covey, in The 7 Habits of Highly Effective People, says to begin with the end in mind. Jim Collins says that one of the biggest things keep good companies from becoming great companies is that they don’t have a clearly established and communicated “hedgehog concept” (his term for vision) in his book, Good to Great. Dave Ramsey, financial guru, radio personality, and author of The Total Money Makeover, says that the best way to ensure the financial success of your savings plan is to establish a clear purpose for the money you’re saving.

Remember that when you’re working with software developers, it’s very important to communicate the vision of your product. A good software development team will make much better decisions and ensure a much higher chance of success if they understand your goals.

At the end of every sprint, scrum teams demo the features they implemented that iteration. As the manager of several infrastructure teams, I can tell you that sitting through a demo of the new data layer for the API (hand-typed telnet operations and all) is about as boring as it gets. Not only that, but the data layer itself provides no value to the customer. Yes, it’s very important, but the customer would never provide us with a story that read, “as a user, I want the data layer abstracted using Hibernate so I can decouple the business logic from the data store”. Unfortunately, developers seem wired to think horizontally instead of vertically.

One method we can use to change this development style is the tracer bullet. I first read about the tracer bullet in The Pragmatic Programmer by Hunt & Thomas, and then again in Agile Estimating and Planning by Cohn. The gist of the tracer bullet is to think about the system vertically and program a very small part of the program end-to-end. Don’t confuse a tracer bullet with a prototype; prototypes are meant to be thrown away, while tracer bullets will become part of the production system.

The Tracer Bullet is a fantastic method for getting teams to break out of the waterfall mindset and fully embrace agile.

For more information about the Tracer Bullet, I highly recommend reading this in-depth interview with Andy Hunt and Dave Thomas.
http://www.artima.com/intv/tracer.html