• Better Git Stashing

    I often find myself wanting to stash just one change or file rather than stashing all changes in the working directory. Git hasn’t historically made this easy. Although you can git add the changes you want to keep and then run git stash --keep-index to stash the other changes, it is rather clunky.

    Thankfully there is a better way.

    Git has an interactive stash mode you can activate with —-patch. It provides the ability to step through code changes and stash (or skip) each individually (git add has an interactive mode as well).

    » git stash --patch
    diff --git a/content/post/purge-redis-keys-with-lua-script.md b/content/post/purge-redis-keys-with-lua-script.md
    index a5f9fc5..78069a8 100644
    --- a/content/post/purge-redis-keys-with-lua-script.md
    +++ b/content/post/purge-redis-keys-with-lua-script.md
    @@ -6,6 +6,8 @@ slug = "purge-redis-keys-with-lua-script"
    
     +++
    
    +New change here!
    +
     Redis has some powerful [Lua scripting capabilities](https://www.redisgreen.net/blog/intro-to-lua-for-redis-programmers/). One of the uses I've found for this feature is purging cache keys. On occasion I need to purge a set of keys that all have the same prefix.
    
     The following command will do just that.
    Stash this hunk [y,n,q,a,d,e,?]?
    

    Git will display the change along with the surrounding text. If you enter y and hit enter git will stash the “hunk” and move to the next one. You can skip hunks with n and quit entirely with q.

    Entering ? gives you more details about the possible commands.

    y - stash this hunk
    n - do not stash this hunk
    q - quit; do not stash this hunk or any of the remaining ones
    a - stash this hunk and all later hunks in the file
    d - do not stash this hunk or any of the later hunks in the file
    e - manually edit the current hunk
    ? - print help
    

    Additionally there is a new option in git v2.13 that allows you to stashing a file or directory by specifying its path. The simple trick is use git stash as normal but followed by a -- and the path of the file.

    » git stash -- content/post/purge-redis-keys-with-lua-script.md
    Saved working directory and index state WIP on master: 0cba4a5 Hide timestamp on pages
    ------------------------------------------------------------
    »
    

    Finally, if you’re like me and use git stash excessively you’ll end up with many stashed changes you’ve abandoned and no longer need. View your stash list with git stash list and you’ll get something like this:

    stash@{0}: WIP on master: 88dd4dd3 Merged in cst-11221/refactor-cloud-subscription-search (pull request #292)
    stash@{1}: WIP on jest-bug: ad4873d7 Setup StaffId authentication for smoke tests
    stash@{2}: WIP on CST-12687-pdf-link-sidebar: 25b3273c CST-12687 Fix tests
    stash@{3}: WIP on ITIW18Q4-5-setup-cypress: d0d14d03 Simplify tests
    stash@{5}: WIP on master: ed3aac37 Merged in cst-12175/show-upcoming-bill-changes-tab (pull request #235)
    stash@{6}: WIP on master: 505e0278 Merged in remove-max-result-limit (pull request #234)
    stash@{7}: WIP on master: 505e0278 Merged in remove-max-result-limit (pull request #234)
    stash@{8}: WIP on statsd: 37284e82 Setup a statsd client and add some basic instrumentation
    stash@{9}: WIP on CST-12027-account-disabled: 0b7e5cac CST-12027 Add account disabled badge to contact page
    stash@{10}: WIP on master: bf900e1e Merged in CST-12047-backend-uat (pull request #207)
    stash@{11}: WIP on CST-12027-account-disabled: 0b7e5cac CST-12027 Add account disabled badge to contact page
    stash@{12}: WIP on parallel-pipelines: d85a4961 Add node cache back to pipeline. Oops...did not mean to remove.
    

    Use git stash clear to delete your stashes. Be careful to not delete anything you actually need.

    Since I use git stash a lot I’ve created the following aliases in my .zshrc file.

    alias gsp='git stash push' # Push is implied with `git stash` but push allows a file to be specified without -- separator
    alias gspi='git stash push --patch'
    alias gspp='git stash pop'
    alias gspc='git stash clear
    
  • Mock Services with Mountebank

    The Problem

    In today’s micro service world it is common for a service to rely on many others. To effectively test a service we often need more than just unit tests but also need to test the interactions with dependencies. Unfortunately that can be difficult. It might be easy to standup those service dependencies in a test environment. Or it might be hard to populate with realistic tests data. Or they might be unreliable which could result it flaky tests.

    Solution

    To get around the above problems we can mock our service dependencies. My team explored a number solutions but arrived at using a tool called Mountebank due to it’s level of activity, feature set, simplicity, and our team’s existing comfort with the JavaScript ecosystem.

    Mountebank allows our team to create mock services that our app can use instead of the real service. The mock service can be configured to return a predefined response or proxy to the real service and record the response.

    Imposters, Stubs, and Predicates

    Mountebank centers around Imposters. An Imposter defines how a mock service should work. They are defined in JSON and can be set via a configuration file or through a rest API that is available once Mountebank is started. An Imposter contains one or more Stubs that define how to handle requests to the service mock.

    The following Imposter instructs to Mountebank to respond to http requests on port 4547. So far we haven’t defined any Stubs so requests will always get a 501 response (notice the defaultResponse).

    // I'm an imposter and I will fool your app
    {
        "protocol": "http",
        "port": 4547,
        "name": "service-foo",
        "defaultResponse": {
            "statusCode": 501
        },
        "stubs": []
    }

    Next up in the configuration is the Stub. The Stub defines how to respond to incoming requests. A Stub uses Predicates to define rules that requests must match. If all the Predicates in a stub match then the responses are returned. Notice responses is an array. If multiple responses are defined in a Stub it’ll cycle through them for each request.

    The following Stub it is looking for a GET request and a path that matches the regular expression. If they don’t match it’ll go to the next Stub defined in the Imposter or the defaultResponse if none match.

    {
        "predicates": [
            {
                "equals": {
                    "method": "GET"
                }
            },
            {
                "matches": {
                    "path": "^/users/.*/permissions"
                }
            }
        ],
        "responses": [
            {
                "is": {
                    "statusCode": 200,
                    "headers": {
                        "Content-Type": "application/json"
                    },
                    "body": "{ \"foo\": \"bar\" }"
                }
            }
        ]
    }

    The above Stub would match GET /users/1234567/permissions and return a response with a 200 status code and a simple JSON object. JSON bodies in Stubs must be stringified. We can improve the Stub by referencing a JSON file and letting Mountebank stringify it for us.

    "body": "<%- stringify(filename, '../fixtures/service-foo/user-permissions-200.json') %>"
    

    Now Mountebank will return the JSON defined in the file.

    Mountebank has many Predicate types. You can match on nearly any part of a request. For example, in our Hydra Imposter we have a Predicate that looks for a particular key to exist in the request body.

    {
        "jsonpath": { "selector": "$..field" },
        "equals": { "body": "id" }
    }

    Through these Predicates you can build up complex queries to match nearly any kind of request.

    Potential Problems

    While mocking service dependencies aids in testing there are a number of risks.

    Out of sync mocks

    As with all mocking there is a risk of becoming out of sync with the actual implementation. Our hope is to mitigate this in a number of ways.

    1. Rely on contract testing where possible to validate our mocks. This would allow us to validate each part of our app in isolation without relying on brittle end to end tests as much. There are tools such Pact that can enable this.
    2. When contract tests are not a viable options run API and UI tests against staging and production (Post Deployment Verification).

    Complex Imposters

    There are many ways to configure Imposters and if not careful they can become extremely complex. Here are a few observations I’ve made as I’ve used Mountebank:

    1. Focus mostly on mocking happy paths and common problem areas. It is possible to mock out every edge case but that could mean the Imposters likely getting bloated and confusing.
    2. Simplify Predicates as much as possible. Use only those that are absolutely required.
    3. Break up Imposters, Stubs and responses into separate files. We’ve broken the configuration into a file per Imposter (service) and then each Stub includes the necessary JSON files. It keeps the config organized and easier to follow and add to.
    4. Decide on common identifiers that can be used to easily simulate a response. For example, an ID of 11111 (GET /users/11111/permissions) could be used to always return a 200 response but an ID of 22222 could always return a 404. An ID of 33333 could result in a 500 error code.

    Future

    Our next step is to continue to build out more tests that rely data provided by Mountebank. Additionally we are investigating Post Deployment Verification and Contract testing as noted above. As usual the higher you get in the testing pyramid the more difficult and brittle tests get so we’re still figuring what to test and how much to test.

    I’m always curious to see how others do testing so please get in touch and we can chat!

    Additional Documentation

  • Unbreak Your Code With Git Bisect

    git bisect is a helpful tool to assist you in finding which commit introduced a bug.

    In a world without git bisect when you need to find the commit that introduced a bug you’d likely have to go through commits one by one, manually checking each one out and then doing some testing. git bisect can make that process a bit less painful by automating a binary search of your commits.

    To start it just execute git bisect start.

    The latest commit likely has the bug so mark it as bad: git bisect bad.

    Now give it a good commit before the bug was introduced: git bisect good 7df148d3.

    git bisect will now cycle through the commits. For each one you’ll need to mark each one as good or bad. Then it’ll move on to the next commit.

    Here is an (abbreviated) example below:

    » git bisect start
    
    » git bisect bad
    
    » git bisect good 7df148d3
    Bisecting: 35 revisions left to test after this (roughly 5 steps)
    
    » git status
    HEAD detached at 4f5d1140
    You are currently bisecting, started from branch 'master'.
      (use "git bisect reset" to get back to the original branch)
    
    » git bisect bad
    Bisecting: 17 revisions left to test after this (roughly 4 steps)
    
    » git bisect good
    Bisecting: 5 revisions left to test after this (roughly 2 steps)
    
    » git bisect bad
    f01f1b96feed92b6afd60ebac576571a98bcda56 is the first bad commit
    commit f01f1b96feed92b6afd60ebac576571a98bcda56
    Author: Joe Dev <jdev@companyfoo1000.com>
    Date:   Tue Jan 16 15:40:46 2018 -0600
    
        Add super awesome feature that won't break anything
    

    Each time you mark a commit as good/bad it moves to a new commit. You can see the git status above shows the current commit. After the last git bisect bad it identified the commit that introduced the bug.

    With the commit found just clean up with git bisect reset and you are good to go!

    There are quite a few other options with git bisect that you can read about in the git docs.

  • Level Up Your Git Skills With Aliases!

    Like many programmers I use git constantly. And also like many programmers I get tired of typing the same commands over and over and over. So I’ve added the following aliases to my .zshrc to make common commands easier and quicker to type. You could do something similar via .gitconfig or .bashrc.

    Some of my most used aliases are gpb for pushing a branch, gca for committing all changes, gs for getting the status, and gup for rebasing from master.

    Let me know if you have any suggestions!

    # Git
    alias g='git'
    
    # Shorten common commands
    alias gst='g status'
    alias gpl='g pull'
    alias gpu='g push'
    alias gf='g fetch'
    alias gco='g checkout'
    alias grb='g rebase'
    alias grbc='g rebase --continue'
    alias grba='g rebase --abort'
    alias ga='g add'
    alias gm='g merge'
    
    # More succinct status
    alias gs='g status -sb'
    
    # Show all branches in the order of last change
    alias gb='g for-each-ref --sort=committerdate refs/heads/ --format="%(HEAD) %(color:yellow)%(refname:short)%(color:reset) - %(contents:subject) - %(authorname) (%(color:green)%(committerdate:relative)%(color:reset))"'
    
    # Nicer diff
    alias gd='g diff --word-diff'
    
    # Delete a branch
    alias gdb='g branch -D'
    
    # Fancy git log
    alias gl='g log --graph --stat --abbrev-commit --date=relative'
    
    # Push branch and set the upstream
    alias gpb='g push --set-upstream origin "$(g rev-parse --abbrev-ref HEAD)"'
    
    # Safer force push
    alias gpbf='g push --force-with-lease'
    
    # Pull the latest from master and rebase
    alias gup='g pull --rebase origin master'
    
    # Pull the latest from master and merge
    alias gum='g pull origin master'
    
    # Ammend a commit
    alias gam='g commit --amend'
    
    # Save changes temporarily
    alias gwip='g add -A && g commit --no-verify -m "WIP"'
    
    # Undo the last commit
    alias gun='g reset HEAD~1 --mixed'
    
    # Super undo using the reflog...careful. Undo your undo.
    alias gungun='g reset "HEAD@{1}"'
    
    # Delete all merged branches
    alias gcl='g branch | grep -v "master" | xargs g branch -d'
    
    # Commit
    alias gc='g commit -v -m'
    
    # Add all and commit
    alias gca='g add -u && g add . && g commit -v -m'
    
    # Create new branch
    alias gcob='g checkout -b'
    
    # Add all
    alias gaa='g add -u && g add . && g status'
    
    # Interactive adding
    alias gap='g add -p && g status'
    
    # Interactive rebase for squashing commits
    alias gir="g rebase -i origin/master"
  • git merge --squash

    I’m a fan of rebasing in git. I like being able to make lots of small, messy commits without worrying about a readable history and cleaning it up later. But on occasion I’ve had issues where rebasing and squashing my commits has resulted in many merge conflicts. I’m not really sure why this sometimes happens but it is a pain. Fixing merge conflicts can be error prone and time consuming.

    Thankfully I’ve recently come across git merge --squash which can help in these situations. To use it do the following:

    # Checkout master or whatever you branched off
    git checkout master
    
    # Create a new, clean branch
    git checkout -b fancy-new-clean-branch
    
    # Now merge the old, messy branch into the new
    # branch with git merge --squash
    git merge --squash old-messy-branch
    
    # This will copy all the changes to the current
    # branch but will NOT create a new commit. Add all
    # the changes and then commit.
    git add .
    git commit -m 'New easy to understand commit'

    Now your git history on the new branch will have a new single commit that you can push up and create a pull request for.