Extended cognition: a view for product development tooling?

This very interesting article talks spiders offloading some cognitive tasks into their webs.  Sounds strange, but what if you think about the web as a purpose-built extension of the spider's own senses.  The spider builds a web that delivers information about what has landed in the web. The configuration of the web modifies the sense input to the spider sitting on the web.  I can imagine some wevb structures helping to perform filtering around something landing in the web vs. wind.  And is the thing that has landed alive or a bit of stick or a leaf?

The concept explored here is Extended Cognition which is the view that mental processes (mind, thinking) can be extended beyond the brain/body and into the environment around.  I guess you could claim a lot of human-made culture is like this. 

This is an interesting alternate view. Given all our product development tooling: charts, graphs, processes, meetings, lists, spreadsheets for example.  How is it when we look at all this stuff through the lens of Extended Cognition?  Is there a useful view in there?

I'm just going to ask the question for now.

Frameworks are artefacts of a conversation

In Six tips on front-end framework selection I talked a bit about how to go about choosing a front-end framework for a new project.

So, what is the bigger picture here. What are frameworks all about?

I think that frameworks are solidified artefacts from an ongoing conversation we are all having about how to develop products more effectively, easily, cheaply, or whateverly.

Artefacts from a ongoing conversation.

Over time, we develop stuff, and aim to improve the how of the development. In this process, people solidify ideas into frameworks at various points. They meet some shared pain or add an exciting capability and a frameworks becomes popular. People try it out. Their learning feeds the minds of those building the next framework: introducing new metaphors, tools, style and syntax as they go.

These artefacty frameworks also contain reactions to the environment around — available browser performance or particular browser features lead to new ideas being tested in a framework.

That is all I wanted to say: Using, testing, moaning, exclaiming about a framework are all part of the conversation we have to have. New frameworks follow old ones. The are built from previous experiments and ideas.

Why does this matter? Perspective helps make good decisions.

You could deny it, but every front-end framework is thus made of jQuery. From the jQuery experience, annoyances and inflexibility we all dealt with, we have the beginnings of the modern browser framework. jQuery informs AngularJS. AngularJS informs React. And so on.

Chat is not idle: keep your history

Both Slack and HipChat - and probably most competitors too - have these tasty free plans that have some limited amount of searchable message storage.  Fine.  This gives you a free way to get started using a chat tool.  Okay so far.

Now, here's the problem:  At some point you and your team will get beyond the free limit and you'll ask management to pay for the app and they'll look at the pricing and not quite manage to pay up.  Everything keeps working on the free level, so it all looks okay, but what is happening is actually pretty bad for your business.

You've got limited message storage.  Each new line of chat means that some previous useful how-to or solution or web address has disappeared forever (unless you pay up later).  So, you know that host name or phone number or set of steps is stored somewhere in the Slack, but it has gone.  So you ask that question again, and get answered again (with time wasted in the process), and so it goes on.

Losing this information is costly.   There's an unwritten contract that stuff in chat is still there, and we've violated that. We're keeping water in a leaky bucket and people sort of know that but forget.  And having a chat system handy stops people putting stuff in searchable email or a wiki or confluence or suchlike.  So, this is a double fail now.

Chat without history is expensive and time-wasting.  Beware.  Get a buying commitment early.

Six tips on front-end framework selection

React or Angular, or VueJS, or whatever?  What do I choose for this next app I'm building?

Framework selection isn't easy. Here I want to put down my thoughts on what I think matters.

1. Complex things are complex

Firstly, let's be honest about trivial examples. They are trivial.  Nothing I've encountered in building web apps is anywhere near as simple as a To Do app.  TodoMVC is lovely for comparing framework features BUT any non-trivial app will push the boundaries of the framework and build process and comparing via a simple SPA comparison won't be that helpful.

No matter what you choose there will be always be too much boilerplate code and too many untidy corners where it doesn't work nicely. One or more of the framework's metaphors will suck.  That will all just happen.  So, rather than look to the framework to solve your complexity, look how it can support you through it.  

How does it scale to many many components and pages? How easy is it to test within and write tests for?

2. Fashion

Fashion drives a lot of our choices.  One of the wonderful things about the web industry is that there is plenty of new to play with, and there is an ongoing 'conversation' between all the developers on how to build web pages and web applications.  As a part of the conversation new tools are developed, languages are enhanced,  and new ideas emerge.

There is a massive temptation to try the latest thing, and the latest thing sells itself as the best yet.  However, these are all steps in an ongoing conversation we developers are having about how to do things. New frameworks and new versions of existing frameworks are really just a few more sentences in the conversation.  There will be something more amazing next week.  

So, keeping the choices in this sort of context really helps with decision making.  It is worth calling out the fashion aspect as you are deciding.  At least be honest if you just want to do the latest thing :-)

3. Nothing lasts that long

Ideas get old. Things stop working. Support evaporates.  As a part of this drive to the future with tools and frameworks, this will happen to your favourite framework, language, operating system.  When it happens it may require you to upgrade, mothball, re-build etc.  This will probably happen sooner then you expect and will be complicated by dependencies. 

Especially: don't expect stuff to just sit in npm for ever.  Stuff will evaporate.  Have a plan.

4. Skills

I've just finished a re-building an app into Angular 2 from earlier Angular 1 app with a weird backend Java CMS.  Moving to Angular 2+ pretty much mandated TypeScript instead of JavaScript.  Now, guess what happened?  TypeScript typing looks a lot like Java - and it became pretty easy for the Java devs to join in the front end dev using TypeScript. That was unexpected and really helped the project along.  

My point here is:  what skills do you have, and therefore what do you choose?   Might be fun to take on the new thing, but it you are therefore niche-ing into tech that has little support from the people around you, or if it is nice or fashionable enough it is going to be hard to hire people.    The easier and less exciting path (which includes leaving work earlier each night) might be to pick something better supported around you, rather than the bleeding edge.

5. Appreciate simplicity, plan for difficulty; read the source

Framework simplicity is valuable; a framework that simply solves your problem is golden.  Given that your non-trivial application will get complex or have some sort of twisty corner in the URL scheme that doesn't quite work with the router, you'll end up beyond the documentation reading the source.  So why not start now?   I encourage you to code review at least some of the whole framework before you start using it.  You'll end up looking in the source anyway, so why not use this as a part of your evaluation.

6. Or just don't

Finally, maybe you don't need much framework, and maybe none.  If your page is free of routing, relatively simple, then why not consider the no-framework approach?  However, you are going to need good patterns and coding standard and testing or you may well create an organic mess.  Remember, a framework is holding you to certain patterns, so you'll need to decide on and apply your own to keep stuff maintainable.

I hope this is useful. Feel free to comment below on your framework choices, or contact me if you'd like some help or chat about framework selection.

My recent framework selections (in reverse time order) look like this: Angular 2+, No framework, Angular 1.5 + material, No framework, Angular 1.x.

Technical Debt - 4 steps to take control and profit

When planning or story grooming starts, developers are always making these little side comments about Technical Debt, or simply Tech Debt -- the ongoing future cost of doing a quick hack rather than building something properly. This is usually done with a kind of lament and then forgotten about again and again until it is time to abandon the development and start again.

Rather than becoming some wistful lament, I think Tech Debt is our friend.   With awareness of it and a little thought, we get a powerful tool and view to help planning and simplifying product development.  Read on to find out more about what we can do and grab a quick template to make it easier.

What is Tech Debt and why do I care?

Tech Debt is the ongoing future cost of building something expediently -- to do it quicker.  Rather than solving a product development issue properly, we build something that is an imperfect, partial, or brittle solution.  There’s an initial cost for not solving the problem properly (say, product flexibility or reliability), then ‘interest’ accrues as future development gets more difficult while working around this quick hack.   

Interest accumulates over time. It compounds.  So, either we fix the principal problems sometime or the interest bill gets so high we are bankrupt. This is frustrating for developers and managers alike and is the cause of much damage to morale in longer-lived projects.   Eventually it becomes too much and the decision is made to start development on something new and throw the old one away.  Or possibly the product is just scrapped. This is either expensive or very expensive.

What to do?

So what can I do about tech debt?   Not accruing it is a good place to start.  You’d need to be have perfect foresight to manage that completely, but good and careful design and experience helps a lot.  People who have seen the full lifecycle of a product will get this pretty naturally.

You’ll still accrue some anyway.  Just like we might take a loan sometimes, to expand a company or buy a house.  The debt doesn’t have to be a problem if it is understood and managed.  Managed how?

Managing Tech Debt

  1. Make it visible -- either keep a team-wide or personal tech debt register (say, a spreadsheet) that notes down debt accrued or later discovered. This needs to be organised by component so you can quickly reference what needs reworking.  Bring this along to planning and design meetings so you have the information to hand.  Here's a Google Sheets template you can copy and use.

  2. If you are going to work on some component, then it may be the time to tackle some debt.  So, knowing there is some, it can be built into your estimates and work plans.  You can talk about it like this:  “Given we are going to have to refactor the serial interface, we ought to sort out the shortcuts in the command parser from last month”.  Management has to be willing, but that’s usually easy to sort out if you’ve got a list of debt to point at.

  3. Once you know you have a system for recording the debt and some management support to keep it in control, you can agree to pay off some debt each sprint by taking on a little extra development.  This can be long term sustainable as long as the debt doesn’t accrue too high.

  4. Relax.  You know what the debt is and how to service and reduce it.   Profit.

And that really covers it.  Using visibility of the tech. debt (a team or personal register), and some support in resolving it, we’ve got this Tech Debt problem under control.

Review: Nearly six months with Angular 2

Angular 2 is at rc.5 as I write this.  There is not quite a 2.0 release.

After nearly six months working with the various release candidates, here's my review of using using Angular 2 to largely re-build an Angular 1 app and integrate it into a mixed Angular 1 and 2 environment.    This will give you a high level overview of what is different in building an app for Angular 2 and serve as a launch pad into your own research.

This article will be helpful if you are looking on starting a new project and trying to work out whether to start with Angular 2, migrate from 1 to 2, go React, or something else.

TLDR;   There’s lots of good stuff In Angular 2: Components are first class; JavaScript modules and classes are used everywhere; Promises are replaced by RxJS.  You can migrate from Angular 1 reasonably easily. Write in TypeScript, Dart or JavaScript.  Basically Solid. Expect a few small changes before release. More software engineering than browser hacking.

What is all this about Components?

Each generation of web framework brings something new to help with the more complex challenges we face in software engineering in the browser.  Where Angular 1 arose to answer the question “How do I build a complete fully-featured single-page app?”, both Angular 2 and React both arrive to answer the additional question  “How do I build a complete fully-featured single-page app in a maintainable way, using real software engineering?” Components just seem like the obvious next step as we build more complex interfaces made of many parts.  React does this in a less bundled chose-your-own-bits way; Angular 2 in a bit more complete framework kind of way.

I'd say you really want to build component-based if you are doing anything more than making a quick one page website.  You need a component-based approach If you are looking to re-use bits of your app, work with a style guide, or have more than trivial QA/testing.  

As implemented by Angular 2, a component can contain code (JavaScript, TypeScript or Dart), an HTML Template, and one or more CSS files. HTML and CSS can be inline text in the component source or referenced files that are imported at compile time.  The way compiling and bundling works, you end up with these assets specific to the component embedded in the compiled component itself.   Nice.  Makes bundling easy, makes it easy to work on a single bit.

So, your component ends up with an HTML template string compiled in and if you choose, an array of CSS strings compiled in.  When the component is rendered by Angular 2, the component’s CSS is written to a head style tag and targeted with a unique attribute for this component only.

This does make it nice and easy to pick up a component, satisfy its dependencies and test it as an individual item.

Less a Global Soup

Angular 1 always gave me an uneasy feeling. The lack of real modules in JavaScript coupled with all these named strings, plus the sometimes-dashed sometimeCamelcase naming system left a lot of room for duh! Errors, conflicts and confusion.

This has all changed.  You have to be explicit in component metadata about dependencies, directives to be used in templates, services to be provided, input and outputs.   Up until rc.5 that could mean quite a lot of metadata for a non-trivial component, however this latest release candidate provides (and requires use of) an angular Module abstraction to collect a bunch of services, directives etc together.  This looks good though I haven’t tried it yet.

Less global soup also means you can bootstrap multiple angular apps in one browser completely separately, or even bootstrap Angular 1 and 2 apps separately alongside each other.  It it easy to confident about this with Angular 2 because it just doesn’t appear to leak stuff all over the browser global space.

And, biggest of all, the need to be explicit, especially with the inputs and outputs of components, means the rendering is targeted on what has changed only.  Therefore it is faster.  We haven't measurd it, but it looks much quicker than Angular 1 for the same complex pages.

Using a language: TypeScript or Dart or JavaScript

Going from Angular 1 to 2 is a bit like going from jQuery to Angular 1.   There is a lot more formality in the framework and it feels a lot more like software and a lot less like hacking up something quick in the browser.  This just makes sense as the complexity of single page apps increase, and more and more logic moves into the front-end.  To handle this increased complexity, you have a choice of front-end languages: TypeScript, Google’s own Dart, or JavaScript.

Both TypeScript and Dart are typed languages.  This may be a shock for those in love with JavaScript’s nearly unlimited expressibility and type mutability, but y’know, it does make life easier for two reasons I can immediately see.  One: strong typing takes the pressure off unit tests -- you’ll catch a lot of errors simply by using types when compiling -- all those ‘this property should be in this object’ or 'this function should be called with this' are caught by the compiler.  This cuts down on a lot of boilerplate unit tests that you just don’t need. And secondly:  developers coming from, say, Java, and having a little JavaScript awareness will find particularly TypeScript a pretty comfortable language to move into the front-end with.

The Angular 2 docs samples have been delivered in the order of TypeScript, Dart, and then JavaScript.  So Typescript was an obvious place to start for me.   The Typescript tooling is solid.  It cross-compiles to JavaScript well, has a proper strict linter etc. There were the usual new-language annoyances in the beginning, but the results in the project are very very few in-code bugs we’re debugging in the page.

If you really do want to go with JavaScript -- I would not advise that -- you either need to be using ES6 or later or get very used to coding up module and class patterns by hand.

Going Reactive: Promises replaced by Reactive RxJS streams

The next step beyond promises has been taken up.  Most places where you’d expect a promise to be returned (eg http service calls) with an async. result will now provide an RxJS stream to subscribe to.  At the basic level, this is like a Promise that can return multiple values in a stream.  Like a then called multiple times.  There’s a brain wrench in here, a slight syntax change over promises, and done carefully you can end up with an app that has flexibility in the time domain. Connect it to websockets and it keeps changing as the inputs change.

An early design decision in this app we’re working on was to build a subject-based configuration system using RxJS Subjects.  This means that we can in real-time change configuration inside the app and it will change form as it needs to.   Change detection and publishing the change is all done by RxJS -- all our app has to do is subscribe to and listen for config changes.  We use this config to  propagate view changes, query results and display size changes amongst other things and the app just responds as it happens.   There’s no writing code to make things dynamic as such.   It will hurt the head initially if you haven't dealt with it before. This guide helps.

My Overall Review: The Gut Feel

Too many words already, so a bit of my gut feel to finish off.    This is a good framework and at rc.4 it is solid.  It feels maintainable. It is expressive, in that it is easy to get stuff done in the templates and in the component TypeScript itself.   There is a step up in seriousness -- be ready for that or go for something lighter-weight.  It is fast compared to Angular 1.  The reactive feel is enjoyable.

I’d pick it up for the next reasonably-sized project or maybe even a small one.

Where to go next?

I'd suggest from here you dive into the Angular 2 documentation over at angular.io, and try out the quickstart.

Angular 2 - useful links

Angular 2 is a release candidate: rc.1 as I write this.  The official documents still have some missing pieces.  Here are a collection of useful links and my own usage comments to help fill in the missing bits:

Official Docs

Official latest Angular 2 docs - fairly good. Some bits missing as of rc.1

Official TypeScript docs - very good - starts with basic types selected

Unit Testing

Unit Testing Recipes - useful recipes for testing Components, Directives, HTTP Mocking etc - though think this conflicts with the latest I've found on async injecting into unit tests.  I might be wrong.

Testing Angular 2 Components with Unit Tests and the TestComponentBuilder (RC1+) - -but see this repo for tests actually updated for RC1: https://github.com/krimple/angular2-unittest-samples-rc    --- note that TestComponentBuilder has been moved from @angular/core/testing to @angular/compiler/testing.

Touch events

Haven't tried this yet -- not sure what really works. It does seem that Hammer.js must be loaded before angular2.

Bootstrap Carousel

..I'll add more as I revisit links.. First week or two with Angular 2 involved a lot of reading source and searching for solutions..

The five deep themes in Scrum

Scrum is a reasonably human-friendly process for developing products.  It seems a lot nicer and more effective than waterfall and other traditional project management.  But what is actually going on in Scrum?  Apart from attending a bunch of ritualised meetings, what is going on for individuals in the process?

Here are the five deeper themes I like to work with.  I find that keeping these in mind and using them as a lens to view team behaviour and progress really helps identifying blockages and improvements:

1. Rhythm

Scrum brings the rhythm of the Sprint (typically 2-4 weeks) and the Daily Scrum or standup.  These rituals repeat, marking beginnings and endings for the team. There are lots of beginnings and endings, so very different to a single 'delivery date' that the team might get in a traditional project.  Regular beginnings and endings make it easier to make process changes and see changes happening.

2. Narrative

The Daily Scrum requires a team member to speak about their work since the last one; to form a narrative; a story for themselves.   The narrative is witnessed by others in the team.  This helps the individual make sense of what they are doing -- to make and adjust and reflect on the story.    Similarly, a review and retrospective gives the team as a whole a way of making and processing the narrative of the last Sprint.  They have a story for how it was.

3. Reflective Practice

Making a narrative and a regular rhythm opens up the possibility of Reflective Practice.  Reflective practice is the process of reflecting on events and actions to allow continuous learning and growing.  The team can do this regularly within Sprint Retrospectives.  Identifying changes; trying them out; witnessing and reflecting on the results.    This is where the team and individuals can grow out of their normal roles.

4. Ownership

The team estimates the work and decided on how much they will take on as a part of Sprint Planning.  This gives the team ownership of their own work, setting up responsibility, bonding the team on a shared purpose shared control over what they do.  This greatly helps to generate ownership and responsibility for the product they are delivering.

This is not automatic. As a Scrum Master, I have to be aware of this during Sprint Planning, make the space and allow the team to take ownership.

5. Direction and Purpose

A well-defined Product Backlog and a stable current Sprint gives the team a direction and purpose. This is context setting for current work and builds a stable base.  We want team members to be both diligent and innovative; they needs a stable world to work within.

Careful planning and an accessible and decisive Product Owner allows the team to plan for current reality and quickly resolve any hanging uncertainties that undermine purpose and direction.

Which agile tracking tool?

This is a pretty common question:  What tool should I use for tracking my Agile project or Sprint?  

In my humble opinion, based on what I've been using recently:

If you want a simple display for a few tasks with easy setup and easy start, go with Trello.  This is really a visual card column system, Kanban style.  It is beautifully ad-hoc, and you get a good usable tool quickly.  Create a board, add some columns, add some cards.  As easy as that.   I find the limits come when you can't fit all the cards on the screen.  It also does collaboration surprisingly well.  A favourite quick planning tool.

If you're looking for something that definitely pushes you into thinking in Scrum-like Sprints, I'd go with Pivotal Tracker.  It has opinions about how you'll plan and work.  It organises sprints for you, helps you build and manage a backlog easily.  I  like this and use it for a lot of my personal and small projects. Pivotal manages sprints and velocity fairly seamlessly and otherwise stays out of the way. 

If you've got time for configuration and are on the bigger project side of things, I'd take a look at Atlassian Jira Agile.  You get something that is more like a issue tracking system with reasonable add-ons for agile processes.  It works well when configured.  You will spend configuration time.

And if none of these sound like you: start with an on-line collaborative spreadsheet (e.g. Google Sheets) and use that.  If the point is to plan and communicate, you can go a long way with a single spreadsheet with a few tabs. 

There are a lot of tools in this space.  Ask Google if you want to go deeper.

Or give me a shout and I can chuck in some more ideas.

Scrum retrospectives: what is going on, how to make them work

Scrum is a beautiful thing.  We carve out a corner of the business-as-usual world and change the values and working ways.  We do this because it is one way we can get a product built in the chaos of modern corporate life without sacrificing people, a ton of money and the quality of the product.  

And I’d say the Sprint Retrospective, and what follows from it, is the most beautiful part of all.  Enshrined in the process, every two or four weeks, we pause for a moment and look at how the last sprint went. We’re looking at our conditions. Our scrum.  Our process.  And then we look at how we can improve them, and make a plan to make some improvements over the next Sprint.

This is reflective practice, done as a team, regularly.   And this is really important.  With a reflective practice, the team and team members are able to learn, grow skills and develop.    There are opportunities for people to try something different, design a part of the process, step up to something, and be witnessed doing it by the team.

In this way, the sprint becomes a place for learning and experimentation, for growing skills and virtues in the team members.

So what does this mean for running end-of-sprint retrospectives?  Here are some reflection points I use before beginning a retrospective:

1. The retrospective is a way for us to improve our ways of working, contentment and conditions.  It is important for team morale and for individual’s growth.

2. As a scrum master, I'm looking to embed a culture of continuous improvement via the sprintly reflective practice and doing work in the Sprint.  

3. Key messages for scrum master and team going into a retrospective:

  • The team has the power to improve their conditions (improve their processes and work lives, deliver more points, improve quality).
  • Team morale comes from delivering points and working on improving conditions. Both are important. Skipping either will lead to trouble.
  • Blaming others is not going to work.  Striving to understand them is a beginning.
  • When evaluating the results of changes made from retrospectives:

4. When working on improving conditions, failure is totally OK.  Focus on the learning from what happened.  This is a kind of gentle inquiry into what happened.  Hold it lightly.  We’re looking for this kind of feeling:  “[laughs]. Wow I really screwed that up [laughs and smiles]. I wonder why?”.

5. Blame destroys the learning.  Work away from “They screwed it up!” to a gentle inquiry into what happened and why.  Understanding the context of the world/business we are in helps us work out what to do.

6. Witnessing is important. Team members need to witness each others successes and failures, doubts and concerns.  Therefore the importance of showcasing both functional work and retrospective-driven improvements.

7. Sometimes, a team is too burned or shocked or harried to be able to get in a place to run a retrospective.   It is important to notice these times.  Rather than push it, here is an opportunity for some time out.  Ideally use this time to go out for a walk.  Nature helps if available. Get out of the office and just walk for a bit.  People will chat amongst each other.  I don't feel any need to structure this, it seems like people chit chat, work and team issues come up and are discussed and put away again.   A camp fire would probably work even better, but that is harder to organise in the daytime in most urban areas :-)

8. As a leader, show your weakness, acknowledge your screwups, and showcase your learning with the team.

OK, that is pretty dense.  That is definitely enough to think about. Distilled to one key point:

  • Cultivate a gentle, inquisitive, reflective learning amongst the team and yourself. Hold it all lightly. Laugh, but with kindness.  Keep the rhythm.