When I started at Intuit about a year ago, one of the first things that struck me was how little technology we open sourced. There was so much cool stuff that was being built, and so many unique problems we we’re working on, however we were still operating as a product company and every line of code was automatically viewed as proprietary. One of the reasons I was hired was to help Intuit transform from a products to a technology company, so one of my first tasks was to tackle open source, or the lack there of.
Turns out, we actually had a really good checklist for releasing something into the wild (which I will go over later) but, I found the reason we didn’t really release much was because our senior leadership didn’t really see the benefit and our organization was not structured to work in an open source fashion. With the help of my boss and the TurboTax chief architect Alex Balazs, our Senior VP Nat Natarajan and our tech evangelist Aliza Carpio I started working with our engineers and senior leadership to change this. If your company is already hip on open source, feel free to skip down to the Checklist section, but for the rest of us enterprise grunts, read on!
First, we really had to articulate what the benefits were of open sourcing a piece of code. The most accepted benefit of releasing something is brand recognition and talent acquisition. The reality of the market today is that there is way more work than there are engineers, so companies are constantly fighting for talent. Intuit is one of the best companies in the world to work for, but ask any engineer what we do and, at best, you’ll hear about how we build tax and business software. BORING! I had the same perception before I came on board and it actually took a good friend and old colleague strong-arming me into doing a technical interview before I was sold.
Every company that did interest me did it with their open source projects. Facebook’s, for example, open graph interested me, as did Google’s Polymer and Angular projects. Hell! Coming from an IT background, even all the cool stuff Netflix has open sourced gave me a good insight into how and what they did and I liked it. Intuit needed to put 5 architects in a room who presented me with the current problem they were working on and we had a 2 hour heated discussion before I said to myself “ok, this place is a great tech company.” Recruiters often think it’s salary and benefits that get the talent, but the reality is the truly great engineers of this world really care about the work over everything else and open source is a great channel to reach out to them.
Recruitment and brand recognition are not the only reasons to go open source. There are a slew of technical benefits that become apparent once you start architecting software that may be used by the general community. First, it forces you to think about more than just your own use case. For example, you’re building a blogging platform and you’re doing it on AWS leveraging RDS. But what if users want to use MySQL or Oracle? I know this is a simple example, but the reality is that when we’re writing code to solve a particular problem, often times we make concessions or take shortcuts that introduce tight dependencies in the system. Developing something that lives in the wild forces you to really think beyond your immediate problem.
The second thing you get is technical feedback. As I said, working on a particular problem we end up heads down with blinders on, however there is no way of knowing who will clone your repo and how they will use your code, but once they do, you start seeing how your code can evolve. I once read an article that described what we do more as gardening rather than engineering so following that train of thought releasing software into open source is akin to taking a plant from the little clay pot sitting on your porch and planting it in the middle of a forest.
Finally, there is always the free labour, but not the kind most Directors think. By presenting the possibility of having thousands of engineers using and contributing to your code, most Directors jump to the idea that software will start to get written for free. In reality what you get is a lot of people telling you why your software sucks. Now, you can choose to be upset by this or you can realize that you already (or at least I hope you do) employ and entire department of people to do the same: your QA department. The previous two points I made really culminate with the third. There is no more valuable asset a company can have than having engineers with a wide variety of skill sets, backgrounds and current projects looking at your code and finding ways it can be made better. Hell…every once in a while, you may even piss someone off enough they actually fix your code. But for the most part, feedback is almost more valuable.
Well, it turns out convincing our senior leadership that open source is a worthy endeavor wasn’t that difficult. After a handful of meetings, people were on board with the idea in general. Where we had the real problem was in execution. Before we could open source anything, we had to make it open source-able, however organizationally, Intuit wasn’t really used to working this way. There was a silo mentality at play that fundamentally made developing any kind of sharable code challenging. Teams were being tasked with, and held accountable for, delivering experiences, or better yet, vertical slices of the product. This led to a mentality of both “I solve for me” and “over the wall”.
Any line of code that could possibly affect the slice of the product being delivered had to be under full control of the group developing said slice. Anything that was outside that slice was considered over the wall. This led to us using 5 different versions of JQuery throughout our product and many instances of passing data from one slice of the product to another using cookies. In addition, any time the idea of shared components, services, libraries, etc. was brought up, it was quickly considered, denounced for not supporting every single possible use case (never mind validity of said use cases), denounced for being an outside dependency and then quickly dismissed only to be built less effectively by each team. In essence, teams viewed common components as a risk, rather than an opportunity.
To address this, we first had to build out an MVP product that could be shared, and then we went to each team and evangelized it. However, when teams pushed back with the expected feedback, we said “ok, then fix it.” Over and over we implanted the notion that just because a piece of code was not yours, it did not mean that you had to rely on someone else to fix it in a timely manner. We started creating a system of active participation. For each feature that was being shared, we started hosting working sessions, consensus meetings and created multiple channels of communication. We had to make it clear that if you want something done right, you gotta do it yourself and that’s OK.
We also had to address the problem of ownership. In the silo system I described each team has clear ownership of what they were building. If one team was building the UI component and another team builds the service that would support it, you would see a lot of requirements and requests being thrown back and forth…until the service team would just make their own experience and the UI team would make their own service, which means each would “suck” about equally. The problem was that teams were conditioned to only work on things they owned. To counter this we introduced the idea of internal stewardship. Teams would no longer OWN features, capabilities, repos, etc. Instead they were becoming stewards, guiding and growing the code while actively engaging with anyone who wanted to participate. Essentially, if you have ever used GitHub and created a pull request…its pretty much that.
The last thing we had to address is what happens when two teams really have opposing requirements. Well, again if you have ever developed anything and pushed it to GitHub, you know the answer is forking. However the idea has been to try and limit forking as much as possible. If everyone forks and maintains their own copy of shared components - this would defeat the purpose - so we always have it as an option. But, there is someone from the architecture organization actively monitoring forks for ways of abstracting shared features or possibly moving people towards a future merge.
I know I probably painted a bleak picture of Intuit in this section, however, the reality is that a lot of large companies, especially if they have been around for a while, are in the same situation…we are actively getting better every day, so can you. Now that I’ve told you what held us back from releasing software, I will now share the awesome checklist we have to assure when we’re ready to finally open source, we do it right.
The checklist consists of getting 3 sign-offs. Essentially you have to talk to at least 3 people who represent different interests and get their buy-in; the business, the legal and the engineering.
The business sign-off is all about cost vs benefit. This is a conversation you have to have with a business stakeholder. For us, that’s a Director and a Chief architect. These people also become the sponsors for your project. This conversation meant to shake out what you’re building from both a product owner and product development perspectives. The goal is to challenge you as an engineer to think about the consequences of moving a project into open source, as it relates to usage, maintenance and governance.
Here are some sample questions you may be asked to answer:
- Why do you want to open source this project?
- Is there anything like it already out there, and if so what makes yours different?
- What is the risk to our business if you open source this?
- What is the benefit?
- Can/Should anything be patented?
- What is your plan to continue developing this once its on GitHub?
- Are additional resources required to maintain it in GitHub?
- Are there additional support components that need to be built/released to truly make the project worthy?
- How do you see the community using/contributing to this project?
- What is the governance for this project? Is it strict one way or will it be influenced by outside feedback?
- How will you evangelize it? Blogs? Conferences? Road Shows?
- Do you have branding?
- Do you have corporate communications involved?
- Have you evangelized it internally and if so what has been the response?
- Should you release it as a sponsored project under the company name, or a personal one under your own?
The second sign-off is legal. Legal is more straightforward and hopefully you have an open source attorney, if not any kind of technically savvy legal council will do. We assume by this point you know what you’re building, why you want to share it with the world and how you’re planning on doing it. But now, we have to make sure the company is safe from a legal standpoint. The following are sample questions you may be asked at this point:
- Can we be liable? This one is crucial. Is what we’re open sourcing going to open the company for lawsuits. This is mostly handled by the license, however it’s a very important conversation that has to happen, and for legal to feel confident.
- Are any patents involved and if so how do we deal with it?
- What license do you want to use? How much do you wish to enable reuse, modification and adoption of your project? We tend to prefer using EPL (Eclipse Public License) but MIT, Apache, GPL, BSD, etc. are all still on the table barring a long discussion with our legal council.
Finally there is the engineering sign-off. This is usually done by a team of Senior engineers, Architects and QEs, all of whom are not involved with your project and ideally, they are from a completely different business unit at your company. By now, you know what you’re building, how you’re going to support it and you’re confident you wont end up in the middle of a civil suit. Now, you need to make sure what you’re about to release is up to par. We all know that just because something works does not mean it’s good code. It also does not mean it should be open sourced in that condition. If you plan on open sourcing your project under your company’s brand, there are much higher standards to which you should consider.
- What’s the state of the code? Are there any loose ends? You really want the project to be at least an MVP that works before you push it out.
- Is the code clean? Las thing you want to do is release a bunch of code that’s not really doing anything, maybe leftover from a previous version or a proof of concept. Clean it up! You’re having guests over!
- What’s the state of the documentation? At the very least, you should have some general documentation around why, how and what in regards to the project as a whole. Make sure you have enough written down to walk someone through getting that MVP running. Ideally you want any APIs detailed out, all features documented and plenty of inline comments.
- Make sure your code is sanitized. I will say this again MAKE SURE YOUR CODE IS SANITIZED! Once you push code into open source, that’s it. It’s out there forever and the last thing you want is for your database passwords, or network infrastructure or trade secrets to be released as part of your project.
- Do you have good code coverage? People will start using your project so make sure they have a good product from the start and have confidence things will work in a way you’ve promised them it would.
- Is the code modular and reusable? Would you be proud of putting your name on this for all to see?
- What does CICD look like for this project? Are your builds automated? How do you create versions and release notes? These things are very important.
- Is there any infrastructure (registry, services, public facing API’s) you will have to maintain as part of this project, and if so have they been properly scaled out and secured?
So where has all of this gotten us at Intuit? We have a handful of iOS projects with a decent popularity in GitHub, as well as a good amount of tools for dev ops. The latest Intuit Tank™ is awesome load generator that has been critical for TurboTax Online in performance testing and load testing our applications. (Check out Denis Angleton’s piece to learn more.) There is still a great deal of work we need to do at Intuit to become a truly technological organization. Today, we look forward to our future GitHub contributions…we are in the early journey of ramping up.
About Bojan "Boki" Beran
Bojan "Boki" Beran is Web UI Architect in Consumer Tax Group at Intuit Inc. He leads the technical strategy in web development and responsive design for TurboTax Online as well as Open Source efforts and community building. Prior to his current role at Intuit, Boki was Technical Lead at Piksel Inc where he worked closely with R&D teams and developed connected TV applications using Samsung SDK, iOS and Android. In addition, his UX experience allows him to have a unique insight into how web users behave and how to create products with intuitive interfaces. Boki is passionate about learning new technologies, teaching (he has several publications) and collaborating with internal and external communities. Boki is a UCSD grad with a BS in electrical engineering with focusing on artificial intelligence and modern control systems.