The Organized Chaos Approach To Scaling A Team

Here at x.ai, we make an AI personal assistant who schedules meetings for you. Sounds simple. Don’t be fooled. Artificial intelligence is an enormously challenging technical problem.

Six months ago, we faced a conundrum: Our tech team grew in size from about 15 folks to over 30 at a fast rate. We wanted to make sure that despite the humongous growth, we could readily pinpoint what we needed to build and then go build it quickly and well.

Typically, there’s a pretty basic tradeoff to achieve scale and retain agility: As your team gets bigger, each individual’s range of freedom and resulting impact gets smaller. But we didn’t want that.

We wanted an environment where, as we grew, team members still had the potential to have great impact and do the best work of their careers.

At the time (late summer of 2016) we were fairly siloed, with different functions owning different parts of our stack. One side effect of this arrangement was a wide variation in our approach to management. Basically, we weren’t consistently good at prioritizing our work.

So, we tested a bunch of new stuff in an attempt to scale our team without overwhelming ourselves with structure and silos.

First: What Are The Goals?

We had implemented individual and team OKRs as our team grew beyond a tight-knit set of engineers and data scientists. Over time we learned that OKRs, which are tracked over three month’s time, were too rigid given how fast things were moving for us.

Instead, we decided that we should align the team around a small set of quarterly company goals—if any single project didn’t contribute in a meaningful way to a specific company goal (and there were no more than five), then that project wouldn’t be worked on.

These goals framed all of the work we wanted to accomplish. Now, we needed to decide on what specific projects to focus on, how to assign team members to specific projects without replicating our silos, and what other processes might help us scale.

Transparent Discussion Using RFCs

RFCs, or Requests for Comment, became a cornerstone of our process. These highly structured shared documents let anyone with an idea for a new process or approach to our work share it with the team, get comments, and then, if accepted, implement their idea.

These ideas can come from anywhere. A now-classic example is our “night’s watch,” proposed by one of our data scientists. Basically, they observed that we weren’t doing a very good job of addressing critical bugs and worse, we were pulling engineers off important projects in an ad hoc fashion to put out fires. We needed to find a way to put out fires in a timely fashion and protect technologists’ time.

This particular RFC had all the requisite elements:

• A statement of the problem to be solved.
• A solution: A weekly on-call rotation of three technologists.
• Any relevant requirements. In this case, there was a Game of Thrones “night’s watch” theme, complete with vows.

Once our data scientist opened up the RFC (via a Google Doc), anyone in the company could comment. Like all RFCs, this one was open for comment for only two weeks. A brief discussion ensued and within about 10 days, the RFC was accepted. We then implemented the “night’s watch” as outlined.  

RFCs functioned as a first step in implementing new processes. The goal was not to achieve consensus. Rather, we wanted to increase the level of transparency around company initiatives and encourage conversation. And because we used shared docs for all RFCs, that conversation became part of the public record.

Self-Selection Of Managers

We also wanted to make sure that the tech team felt a great sense of autonomy—and a commensurate sense of responsibility—as we grew. And we were sure that any "command and control" style of management (in which tech managers assigned work to technologists) would not give our team the sense of ownership we believe is necessary to ship high quality code week over week.

So we allowed all employees to self-select their own managers, projects, and project leads (we call this self-selection at all levels of the stack). While it may seem radical, we didn’t make this type of self-organization up; we borrowed heavily from Pandora’s prioritization process.

Some core elements of self-organization worked really well for us. Selecting your own manager, for instance, was largely a non-event. It reinforces people’s real freedom to choose their manager (because they could always choose another one).

Selecting your projects also worked well. People generally know their strengths and how they can help the company succeed.

Transparency Via Demo Days

Finally, we instituted a Demo Day to take place every two weeks. There were to be two parts to Demo Day. The first half would be like a mini All Hands company meeting. Everyone in the company would attend, and the tech team could share projects that they’ve completed.

Our Customer Experience team would also share key data about customer acquisition and engagement as well as customer pain (e.g. bugs and pain points coming in through Customer Support). After this, we'd break into smaller groups, and the tech team would select which projects they wanted to work on.

During Demo Day, we’d give everyone in the company “currency.” We used sticky notes and assigned a value to each note—five days of a single technologist’s time. Each person in the company would receive 25 days’ worth of technologist time, or five sticky notes.

Add these up, and the sum represented the amount of time our technologists had to devote to making stuff for that quarter. Which is to say, if our projects took more time to complete than the amount of currency we had to spread across them, we’d have to figure out which ones not to work on.

That’s when the fun began 😜.  Everyone would “fund” the projects they felt merited time and effort this quarter. This would immediately cull some projects (those that didn’t get fully funded). Then, we’d do one more pass of culling, slimming down the set of funded projects once more, based on team members’ arguments (pro or con) and alignment with company goals.

Did It Work?

Overall our new approach is far more collaborative and cross-functional, which means we produce better work more efficiently.

We also multiply the opportunities for people to lead. Today, anyone can lead technical change—regardless of their title or level in our (still pretty flat) hierarchy. This is a huge win for us and individual team members.

We made these changes six months ago, and in the interim, we’ve built and implemented:

• A huge new piece of software (a key part of the system that processes incoming scheduling data);
• We launched two new products (the Professional and Business editions of our AI scheduling assistant); and,
• We implemented a much more sophisticated layer of systems monitoring.

We did this with a team of approximately 40 engineers and data scientists. From that list alone, you might say whatever we did was a smashing success.

Reality, however, is never that simple.

There were losses, too. For example, we failed, badly at first, to develop a planning process that was disciplined, efficient, and truly aligned to company goals.

Instead, we spent many cycles (and far too much time) talking about a bunch of things that would be nice to do... but that we didn’t have time to do. We eventually got “project list bloat,” meaning we had an absurd number of projects in the backlog.

In terms of self-selection, there were deviations, too. Autonomy only works if you structure the process in a way that clarifies constraints, priorities, and commitments. Unfortunately, parts of our planning obscured the relative importance and urgency of different projects.

For instance, some technologists decided to explore yet another machine learning model when we really needed nearly all hands on deck rebuilding a key piece of our infrastructure.

Just Like Software Production, We’re Iterating

Fortunately, we’re fast learners. We quickly saw that self-selection at all levels of the stack requires a team that is well-balanced and mature with respect to negotiating technical aspects of our system, customer and business needs, and even the individual desires of engineers themselves.

We don’t now (and may never) have the perfect balance of skills and maturity, which means we’ll continually tweak our prioritization systems to account for that.

For example, we now prune a proposed list of projects periodically. We’ve also created forums for the Customer Experience team to give us nearly real-time feedback to help prioritize projects.

And once we realized that two-week cycles didn’t map to our actual pace of software development, we created loose groups to work on projects for longer four-to-six week cycles.

We want to emphasize to the team that some of the things we need to build require a more sustained commitment rather than constant course correction. Now our team has a deeper investment in the longer term goals, which helps push projects across the finish line more reliably.

We’re also constantly working on ways to help people grow into the responsibility implied by self-selection. To bring role models into the organization, we explicitly test for leadership and mentorship when we interview candidates. And we coach contributors to select projects that match their skills and interests and that give them opportunities to work with our technical leaders.

One advantage of self-selection at all levels of the stack is that there’s barely any framework there at all. And being light, the framework itself is quite adaptable. As long as we’re continually learning about our team and tweaking our process to support it, every iteration should be better than the last.

Jeff Smith, Wesley Harris, Alex Poon, and Joey Betras also contributed to this piece.

Next: How The "Silo Effect" Is Hurting Cross-Team Collaboration