• Gumption Trap

    From Wikipedia:

    A gumption trap is an event or mindset that can cause a person to lose enthusiasm and become discouraged from starting or continuing a project.

    I learned this term recently and it really resonated with me. I’ve started dozens (hundreds?) of projects over the years only to abandon them eventually. Often times because I eventually realize it wasn’t really a good idea over perhaps because it was more difficult than I originally estimated.

    Flipping it around though, it is helpful to remember the users of our tools/products and figure out how they might get stuck and give up. If you work on a developer tool, does it need better docs, real world examples, better defaults, or a simplified API? If it is something for a non-technical user, are there steps that need to be removed, buttons made bolder for those with poor eyesight, do the docs need to be made simpler with clearer instructions?

  • Better Git Stashing

    I often find myself wanting to stash just one change or file rather than stashing all changes in the working directory. Git hasn’t historically made this easy. Although you can git add the changes you want to keep and then run git stash --keep-index to stash the other changes, it is rather clunky.

    Thankfully there is a better way.

    Git has an interactive stash mode you can activate with —-patch. It provides the ability to step through code changes and stash (or skip) each individually (git add has an interactive mode as well).

    » git stash --patch
    diff --git a/content/post/purge-redis-keys-with-lua-script.md b/content/post/purge-redis-keys-with-lua-script.md
    index a5f9fc5..78069a8 100644
    --- a/content/post/purge-redis-keys-with-lua-script.md
    +++ b/content/post/purge-redis-keys-with-lua-script.md
    @@ -6,6 +6,8 @@ slug = "purge-redis-keys-with-lua-script"
    
     +++
    
    +New change here!
    +
     Redis has some powerful [Lua scripting capabilities](https://www.redisgreen.net/blog/intro-to-lua-for-redis-programmers/). One of the uses I've found for this feature is purging cache keys. On occasion I need to purge a set of keys that all have the same prefix.
    
     The following command will do just that.
    Stash this hunk [y,n,q,a,d,e,?]?
    

    Git will display the change along with the surrounding text. If you enter y and hit enter git will stash the “hunk” and move to the next one. You can skip hunks with n and quit entirely with q.

    Entering ? gives you more details about the possible commands.

    y - stash this hunk
    n - do not stash this hunk
    q - quit; do not stash this hunk or any of the remaining ones
    a - stash this hunk and all later hunks in the file
    d - do not stash this hunk or any of the later hunks in the file
    e - manually edit the current hunk
    ? - print help
    

    Additionally there is a new option in git v2.13 that allows you to stashing a file or directory by specifying its path. The simple trick is use git stash as normal but followed by a -- and the path of the file.

    » git stash -- content/post/purge-redis-keys-with-lua-script.md
    Saved working directory and index state WIP on master: 0cba4a5 Hide timestamp on pages
    ------------------------------------------------------------
    »
    

    Finally, if you’re like me and use git stash excessively you’ll end up with many stashed changes you’ve abandoned and no longer need. View your stash list with git stash list and you’ll get something like this:

    stash@{0}: WIP on master: 88dd4dd3 Merged in cst-11221/refactor-cloud-subscription-search (pull request #292)
    stash@{1}: WIP on jest-bug: ad4873d7 Setup StaffId authentication for smoke tests
    stash@{2}: WIP on CST-12687-pdf-link-sidebar: 25b3273c CST-12687 Fix tests
    stash@{3}: WIP on ITIW18Q4-5-setup-cypress: d0d14d03 Simplify tests
    stash@{5}: WIP on master: ed3aac37 Merged in cst-12175/show-upcoming-bill-changes-tab (pull request #235)
    stash@{6}: WIP on master: 505e0278 Merged in remove-max-result-limit (pull request #234)
    stash@{7}: WIP on master: 505e0278 Merged in remove-max-result-limit (pull request #234)
    stash@{8}: WIP on statsd: 37284e82 Setup a statsd client and add some basic instrumentation
    stash@{9}: WIP on CST-12027-account-disabled: 0b7e5cac CST-12027 Add account disabled badge to contact page
    stash@{10}: WIP on master: bf900e1e Merged in CST-12047-backend-uat (pull request #207)
    stash@{11}: WIP on CST-12027-account-disabled: 0b7e5cac CST-12027 Add account disabled badge to contact page
    stash@{12}: WIP on parallel-pipelines: d85a4961 Add node cache back to pipeline. Oops...did not mean to remove.
    

    Use git stash clear to delete your stashes. Be careful to not delete anything you actually need.

    Since I use git stash a lot I’ve created the following aliases in my .zshrc file.

    alias gsp='git stash push' # Push is implied with `git stash` but push allows a file to be specified without -- separator
    alias gspi='git stash push --patch'
    alias gspp='git stash pop'
    alias gspc='git stash clear
    
  • Mock Services with Mountebank

    The Problem

    In today’s micro service world it is common for a service to rely on many others. To effectively test a service we often need more than just unit tests but also need to test the interactions with dependencies. Unfortunately that can be difficult. It might be easy to standup those service dependencies in a test environment. Or it might be hard to populate with realistic tests data. Or they might be unreliable which could result it flaky tests.

    Solution

    To get around the above problems we can mock our service dependencies. My team explored a number solutions but arrived at using a tool called Mountebank due to it’s level of activity, feature set, simplicity, and our team’s existing comfort with the JavaScript ecosystem.

    Mountebank allows our team to create mock services that our app can use instead of the real service. The mock service can be configured to return a predefined response or proxy to the real service and record the response.

    Imposters, Stubs, and Predicates

    Mountebank centers around Imposters. An Imposter defines how a mock service should work. They are defined in JSON and can be set via a configuration file or through a rest API that is available once Mountebank is started. An Imposter contains one or more Stubs that define how to handle requests to the service mock.

    The following Imposter instructs to Mountebank to respond to http requests on port 4547. So far we haven’t defined any Stubs so requests will always get a 501 response (notice the defaultResponse).

    // I'm an imposter and I will fool your app
    {
        "protocol": "http",
        "port": 4547,
        "name": "service-foo",
        "defaultResponse": {
            "statusCode": 501
        },
        "stubs": []
    }
    

    Next up in the configuration is the Stub. The Stub defines how to respond to incoming requests. A Stub uses Predicates to define rules that requests must match. If all the Predicates in a stub match then the responses are returned. Notice responses is an array. If multiple responses are defined in a Stub it’ll cycle through them for each request.

    The following Stub it is looking for a GET request and a path that matches the regular expression. If they don’t match it’ll go to the next Stub defined in the Imposter or the defaultResponse if none match.

    {
        "predicates": [
            {
                "equals": {
                    "method": "GET"
                }
            },
            {
                "matches": {
                    "path": "^/users/.*/permissions"
                }
            }
        ],
        "responses": [
            {
                "is": {
                    "statusCode": 200,
                    "headers": {
                        "Content-Type": "application/json"
                    },
                    "body": "{ \"foo\": \"bar\" }"
                }
            }
        ]
    }
    

    The above Stub would match GET /users/1234567/permissions and return a response with a 200 status code and a simple JSON object. JSON bodies in Stubs must be stringified. We can improve the Stub by referencing a JSON file and letting Mountebank stringify it for us.

    "body": "<%- stringify(filename, '../fixtures/service-foo/user-permissions-200.json') %>"
    

    Now Mountebank will return the JSON defined in the file.

    Mountebank has many Predicate types. You can match on nearly any part of a request. For example, in our Hydra Imposter we have a Predicate that looks for a particular key to exist in the request body.

    {
        "jsonpath": { "selector": "$..field" },
        "equals": { "body": "id" }
    }
    

    Through these Predicates you can build up complex queries to match nearly any kind of request.

    Potential Problems

    While mocking service dependencies aids in testing there are a number of risks.

    Out of sync mocks

    As with all mocking there is a risk of becoming out of sync with the actual implementation. Our hope is to mitigate this in a number of ways.

    1. Rely on contract testing where possible to validate our mocks. This would allow us to validate each part of our app in isolation without relying on brittle end to end tests as much. There are tools such Pact that can enable this.
    2. When contract tests are not a viable options run API and UI tests against staging and production (Post Deployment Verification).

    Complex Imposters

    There are many ways to configure Imposters and if not careful they can become extremely complex. Here are a few observations I’ve made as I’ve used Mountebank:

    1. Focus mostly on mocking happy paths and common problem areas. It is possible to mock out every edge case but that could mean the Imposters likely getting bloated and confusing.
    2. Simplify Predicates as much as possible. Use only those that are absolutely required.
    3. Break up Imposters, Stubs and responses into separate files. We’ve broken the configuration into a file per Imposter (service) and then each Stub includes the necessary JSON files. It keeps the config organized and easier to follow and add to.
    4. Decide on common identifiers that can be used to easily simulate a response. For example, an ID of 11111 (GET /users/11111/permissions) could be used to always return a 200 response but an ID of 22222 could always return a 404. An ID of 33333 could result in a 500 error code.

    Future

    Our next step is to continue to build out more tests that rely data provided by Mountebank. Additionally we are investigating Post Deployment Verification and Contract testing as noted above. As usual the higher you get in the testing pyramid the more difficult and brittle tests get so we’re still figuring what to test and how much to test.

    I’m always curious to see how others do testing so please get in touch and we can chat!

    Additional Documentation