How I Work

These are some of the software development patterns that have emerged from how I work, where taking certain actions tend to lead to the best outcomes, in the shortest amount of time. Please share any questions or concerns and thanks for taking a look.

Use the feature like a user

For a user facing feature, when I'm assigned a bug and before I write any code, I find it critical to go through the feature as a user first, on each relevant platform, web, mobile app etc. This gets in front of speculation or skepticism about whether the bug can be reproduced, or whether the user took unconventional steps, and helps ensure that any changes that are introduced are based on a solid foundation of what is currently happening (a baseline).

Develop and deploy code in chunks

For feature work that involves database changes, controller code, model code, or front-end code, I try and introduce those chunks gradually instead of all at once. Database changes are sent out first, so we can make sure the change is applied quickly and in place before being referenced by any code.

The feature code can still all be developed at once on a feature branch, but then I'll branch again off that branch and and either git rm files to pare it down to what I want, for example just the database migration, schema file, and maybe a backfill script, or I'll branch off master, and cherry pick the changes from the feature branch, choosing the option that takes less time.

Controller or back-end code may be deployed "dark" initially, then enabled via environment variables, so that it can be easily disabled. Besides the benefits of risk mitigation, by chunking up work, it's also less overwhelming for reviewers to jump in and provide feedback, because they're reviewing maybe a few hundreds lines of changes or less, compared with hundreds or thousands.

We also use a continuous deployment process, meaning code always goes through a continous integration build (the full test suite is run) and a passing build automatically deploys code.

Branch all the time

Create git branches for everything, even simple fixes. There is pretty much no reason to commit on master given how easy branching is. On the flip side of this, maybe once per quarter or at least annually, delete merged branches. I tend to accumulate dozens of branches, so I have the following git alias bds = for-each-ref --sort=-committerdate refs/heads/ --format='%(committerdate:relative) %09 %(refname:short)' and will run git bds|head -n5 all the time to look at the last few branches I committed to, to resume work. Just like cd - will hop to the last working directory, git co - will hop to the last branch being worked on.

Release database changes first

This avoids runtime problems with a database transaction that expects a column to be there, maybe because all the application servers haven't restarted yet. It's also easy enough to manually patch migrations if they don't run for some reason, or revert this commit, without having to revert all of the application code had the database changes and application code been combined into one big change.

Rebase (or merge in master) all the time

If I'm working on a branch, I use rebase to incorporate changes team members are making all the time (or merge master into the branch if I'm collaborating on the branch). Since we dump our database structure to the db/structure.sql used by Rails, this frequently has new database migration changes to incorporate.

If I'm not making database changes on this branch, and generally that is the case (see previous point), then I'll accept the changes and git rebase --continue over and over, and then just check out the master version of db/structure.sql since I don't need anything from it anyway. I also shouldn't have any changes from db/migrations on my feature branch at this point. So I'll generally have 2 commits, the database migrations, and the application code. I'll generally use the bug tracker ID in both commit messages, so that they could be searched and found together later if needed. I have this alias alias gdms='git diff master --stat' set up and check it all the time, to make sure I'm committing only application code files. Before merging the rebased branch, I'll check git diff master from the feature branch one more time, and look for stray unnecessary changes, like whitespace or debugger lines.

Logging

Logging is critical for a number of scenarios, including trying to find unlikely corner case paths by introducing some temporary logs to production, when there is no database row or column data to query on. On the flip side of this, if no one is reading the logs, they are dead logs, and can be removed, which reduces the lines of code and avoids potentially running into storage limits/quotas on a third-party logging service.

Remove dead code

When development is slower, remove dead code (any code that is committed, but not executed), removed dead dependencies (gems for a Rails app), or obsolete documentation. There generally isn't a lot of business value here, but it helps reduce technical debt that make upgrades more difficult down the road. Upgrades are important for performance improvements and bug fixes for frameworks and dependencies being used!

Limit rollouts but keep things simple in the long run

We make heavy use of "feature flags" by setting environment variables that control whether a feature is enabled, or enable it for a limited audience (e.g. "beta" user IDs).

Environment variables are the lightest weight option to achieve this but long term, you don't want to litter your codebase with them.

Environment variables can be used to mitigate risk, but also to reduce operational burden when appropriate, for example by shutting down certain features that aren't critical, but taking a lot of load, when load is a problem.

For example, we have many markets, but they divide into company-owned and franchise-owned markets. These markets have different management structures, and different legal contracts, so we'll occasionally limit releases to one or the other initially. On the other hand, it's important to choose simplicity whenever possible, which means having the same set of features for both market types in this case, to avoid having to support two different paths that are mostly the same.

Unit testing as documentation

Unit tests can be assets, but they can also become liabilities when they are slow, test private methods, or test framework methods. Don't do these things.

In Rails, I prefer using fixtures (over factories, though we use both) and stubbing (mocha) to keep text execution speed fast. However we tend to have a lot of duplication of fixtures with factories given that we use both and the difference is somewhat subjective. I prefer trying to use built-in test helpers like Rails' travel_to (when the time needs to be fixed for test execution) to ensure the test succeeds regardless of the time or day. We run our full test suite on Circle CI so it doesn't block development locally, so build times are a bit less relevant than being blocked locally by running tests, but we also have a limit of concurrent builds running so we don't want to block other team members with long builds either.

Predictable team processes

We use Team Password and share credentials for "dev" accounts on 3rd party services on our team. Generally an individual team member trying out or setting up a new service will follow the work flow of adding the credentials to be shared. We could do more in having a standard way we document new features in terms of their functionality and in how to test them, but this tends to be ad hoc.

We have some standardization on our more formal documentation though, for Product Requirement Docs, API documentation (written by developers) and QA test plans (written by QA and Product), and the predictability here is helpful when kicking off new projects.

Commit early and often

Write good commit messages that have a one line, less than 80 column summary, and then a more verbose description of "why" the change is happening, or "what" is changing at a higher level (the code describes the specifics) and link to bug trackers, google docs, SQL queries, other commits, or any other content that helps describe the context around why this commit is being introduced.

Avoid introducing database changes that cause performance problems

For Postgres, avoid common performance problems, such as specifying a column default, and backfilling existing rows at once, instead do those as separate statements. Another quick example is using the CONCURRENTLY keyword to add an index on any table with a lot of rows. Braintree wrote up this guide on Safe Operations for High Volume PostgreSQL that is highly recommended.

Email management

I create and archive emails with labels daily. Every new email that isn't deleted gets a label, and auto generated emails from bug trackers don't hit my inbox but are archived with a label.

Emails can be archived, but more easily findable under labels (or by searching). Per project labels can be useful to keep messages for a project, archive them, then delete them all, along with the label when the project is shipped. Do not leave emails unread, instead mark them "To do" and archive them and then try and schedule some time to follow up on those. If some time has passed, the window of opportunity may be over, then delete the email. Keep "to do" lists out of email.