How I build software: Enough design to clear my mind

In this series of posts I'm reflecting on how I typically build software using the project I'm working on as an ecologically-valid example.

Following on from the last post where I set out the vision and motivations for my project, now I'll talk about why I needed to design the system and how I started to do so.

As was a theme in the last post, the purpose of design is mostly to clear up my mind – to turn feelings about how the system will work, into visual designs that can be reasoned about, resulting in mental clarity and less reliance on my memory - so I have confidence to start building.

I blogged previously about diagrams if you want more information.

Mental clear out

Even though I had a vision and list of features, I wasn't comfortable to sit down and start coding - too many unknowns. From experience, I know that coding in this state would leave me with scatter brain and lots of erratic hacking. Instead, I wanted a theoretically-plausible design to guide me.

I sketched out a few diagrams and continued to revise them over the course of a week or two. Over the same time period, and often guided by question marks in the design I did research about related topics (such as remote akka).

Let me show you just one example of how the diagrams helped me get a fundamental decision right, and prevented me from spending weeks of my spare time writing unnecessary code. 

Instant payback from the diagrams

One big question I was contemplating was how I was going to transmit messages across the network. My gut reaction (ergo emotion) gave me a strong feeling that json over http was the best option by a mile.

After I created the first version of the diagram I could visually see that I was going to be communicating between two akka applications using json over http. At that point I felt like (especially with the misspelled “you're”):





Like Erlang, akka has it's own protocols and delivery mechanisms for sending messages between remote instances that are part of the same cluster - this is a much smarter solution to my problem. The updated version of the diagram (below) shows this

.

click image to view full size

I just saved myself a LOT of wasted effort (weeks of spare-time coding). Life-lesson: don't trust your instincts or emotions.

You can see other significant decisions on this diagrams that give me lots of time to research and re-consider them before I start to code. Although code can always can be changed, it's cheaper to fix problems the earlier I find them.

A few more diagrams, much more mental clarity

Just to give a full picture of what I considered on this project to be “enough design to clear my mind”, here are the remaining 3 diagrams.

click image to view full size

click image to view full size

click image to view full size

As you can see, there aren't pages and pages of highly-constrained diagrams – just enough to make me feel confident that I can start to build the system with a general outline in mind.

All of these diagrams were created with guidance from the vision, motivations, and high-level traits to ensure they meet the project's goals. For me it would be very difficult to do all of this just from working memory as I sit at a computer coding.

I also find that ideas come to me when I'm in the gym, in the kitchen, or in any other situation where I'm thinking about something totally different. That's good, because a bit of design like this doesn't force me to think about everything when I'm sat at my desk.

Helps bring people on board

These diagrams make it easier for other people to understand the system. Be that people wanting to use it, or people wanting to help me build it (get in touch if that's you). I blogged before about using diagrams to on-board team members.

Often I think to myself how much easier it would be for me to use an open source project if there were a few diagrams about how it worked. So I know there is value here.

I can also use these diagrams to collaborate in other ways, too. For example, If I need help building the system I can use these diagrams as the basis for conversation with other developers.

I'm now comfortable to make a plan and then start coding - exciting

These few diagrams have given me a lot of mental clarity, because I'm comfortable that what I'm about to build, theoretically, should work as well as meeting all of the requirements.

Remember, though, that the design is not fixed. Even better, any time I do make a high-level change I have to update the diagrams. This forces me to see how changes fit into the bigger picture. Down at the code level and without any diagrams, I'm at the mercy of my cognitive abilities to perform all this work.

Having a basic design means I can break up the project and finally start to plan how I'm going to start to carry it out; sounds like my next post.

How I build software: Start with a vision

I'm just starting a new project which is a great opportunity for me to reflect on how I go about building software. Blogging about it means I have to be even more analytical, and also means I can get feedback if I'm lucky.

You'll see how my primary focus is working around the inherent limitations of my cognitive abilities - I take  many opportunities to replace emotion with logical reasoning in decisions making. Also, I try to un-burden my (short, working and long-term) memory as much as possible.

This post shows how I start out on a project by being clear about what I want to achieve.

You might enjoy a quick read of this wiki page about emotion and memory first.

What do I hope to achieve?

If you look at the github repository for this project you’ll see that my vision is:

To create a service bus that facilitates the creation of highly-decoupled (temporal, platform & implementation) business services that can scale horizontally and are highly-resilient to faults.

When I’m designing and building the system I can keep these goals in mind. Any time I need to make a decision, I can use this vision (and motivations below) to guide it.

Without a clear vision I will just have a feeling. Subsequently, this means my current emotional state has a chance to influence the vision every time I think about it. This can result in erratic focus, indecision, or lack of coherence over the life of the project.

Visions are allowed to change. Importantly though, having a vision makes sure that I am logically disagreeing with a written-down statement, not changing goals based on my current emotional state (aka being whimsical). 

What is the motivation?

My studies of psychology again lead me to question my emotions and favour a clear, written list of motivations for the project... I don’t want to feel like this project is a good idea - I want written down logical reasons so I can see a purpose to the project and can enjoy working on it (good use of emotion).

As with the vision, a written list of motivations helps decisions to be more logical. For example, the question: “Should I continue with the project?”, can be logically answered by: “do I still agree with the motivations I’ve stated?”.

Here’s what I have for this project at the moment:

• Want to learn more about building distributed systems and middleware 
• Want to improve my scala and akka skills 
• Need a service bus similar to NServiceBus that works on linux and for any language, for my future projects

What traits should the system have?

I benefit from keeping a small list of high-level properties the system should have which support the vision. Again, these are all about keeping the project focused on solving a certain set of goals. I'm taking work away from my memory and offloading it to a text file.

Importantly I have to think about the traits in more detail too as I write them down. Sometimes they just don't make sense. Other times they conflict. It's not until they are written down and logically evaluated that I notice this.

Here’s what I’ve got so far:

• Highly fault-tolerant - as soon as business intent is captured, it should never be lost. e.g. at least once message delivery 
• Enables horizontal scalability (as much as it can) 
• Easily monitorable (performance of system, errors) 
• Primarily to run on linux 
• Can be used in any programming language 

Todo list and questions list

Any time I realise I need to do something on the project I'll just put it on the todo list. I avoid holding it in memory or losing focus on what I'm doing at all costs.

Any time I'm unsure about something I just pop it in the questions document. For instance "should network communication be platform agnostic or remote akka?". I can come back to it later whilst staying focused on the current task.

What's really important about these simple text files is that they remove the opportunity for my memory to forget the information. Also, just the act of writing down my thoughts forces me to think about them more deeply.

Sometimes, my mind will pick a really stupid answer. But when I write down a problem the obvious solution often stands out. Try it yourself during day-to-day development tasks.

I try to reduce mental workload as much as possible

At this stage of a project I have a good idea of what I want to build and why I want to build it. These reasons are all written down for me to clearly understand over the course of a project – not held in memory as feelings, then re-interpreted through my current state of emotion each time I think about them.

You might think all this sounds like unnecessary work, but for me the pay-off is grand. I work best and feel happiest when my mind is clear to focus on one thing and is not stressed about having to remember important details.

I'm not the smartest developer so I have to get the best out of what I've been given. Hopefully you enjoyed seeing how I try to achieve this. In the next post I'll show I started to design this project and the thought-processes that drive me.

Small things in Scala

Your choice of programming language most likely won't guarantee or preclude success on your next project. But using a language that facilitates fluent expression of your thoughts and feels good to work with can be a positive influence on how efficiently you meet the needs of your customers.

Allow me to show you some of the small, but discerning, touches I've found in my first few months with Scala. I'll leave you to contemplate how they might add-up to productivity on your next project.

Very low ceremony

Scala's perspective is to remove as much noise as possible so that reading code is a smooth experience.

For instance, functions and classes are public by default (else this REPL snippet would error).....

...parentheses and dots are optional for invocation of single or zero argument functions (focus on line 20+)....

...optional type inference for functions and optional return keyword (the last, in this case only, line of the function is the return value)....

…and I really love underscore variables in lambdas – small touch but brilliant (c# you need this!). Lines 4 and 7 are equivalent....

Infix & postfix operations and symbolic function names

We saw how infix operations look nice and clean in the Porsche example above by letting you omit parentheses. It might also read more fluently, and less noisy, in some cases if we could put the arguments before the method e.g. {argument} {method} {object} - the reverse of normal (line 19+).

You can see above how adding items to the plate using postfix operators (the ">>:" function which is a member of the plate instance - e.g. plate.>>:("tomatoes")) and symbolic function names looks really clean and fluent. This isn't a gimmick, but these features should be used with caution.

One example that really impresses is rapture IO that puts well-known symbols to effective use by mimicking the shell.

Everything is an expression

In Erlang, everything (or almost everything) is an expression. I really like that feature, and I'm delighted Scala has it too.

It means that you can make code even more readable. As an example, your if/else statements return the value from the if body or the value from the else body – it's not just for control flow. Have a look...

For-comprehensions and generators

I'm going to...

• declare a function
• iterate over two arrays.
• perform a computation for each item in the first array with each item in the second array
• return the results as a new collection

I'm going to do all of that on one line of code that is as readable as most you will see you in your career.

Scala won't make you 20 years younger...

… but it has definitely given me further insight about how programming languages allow me to express my thoughts, and put a few small question marks in my head about the technologies I want to be working with on a daily basis.

Keep in mind this is just the small touches – I haven't talked about bigger features like pattern matching, some of the much-lauded frameworks like Akka, or the vibrant community Scala has.

As I said to begin with, choice of programming language isn't the defining factor in the success of a project. Nor am I saying C#, Java, or any other language suddenly are obsolete. Keep learning and keep applying critical evaluation is my philosophy - and have fun with it.

Using riak to understand erlang/OTP application structure

If you’re learning Erlang and want to build or understand real applications, or you've heard about Erlang and want to see a few of the concepts that underpin it's ability to let you to create massively scalable, robust,  distributed applications – this post is just for you my friend.

Shortly we will again be digging into the source code of riak - this time to see how a real-world, distributed  erlang/OTP application is structured. This follows on nicely from my last post as we journey down from a deployable erlang package toward its inner-workings.

We will see:

• What state the erlang VM is in when an application and it's dependencies are started
• How the application is actually started
• What the supervision tree is, and how it is kicked off

An application is a promise

erlang applications are often dependent on functionality in other applications – but not how you’re used to in other languages like C# or Java. In erlang, an application’s dependencies are often other applications that actually need to be running - i.e providing a service, and not just re-using code.

Think of it this way: an erlang application is a promise that the instance of the erlang VM running it will also be running all of its dependent applications that need to be running.

It you’re wondering why an application needing dependencies on other running applications is commonplace, just think of it as a way to increase fault-tolerance and provide other distributed-system advantages.

riak’s promise…

In the last post I showed riak’s release script. This contains the instructions about which applications to start. Here’s an extract:


Some dependencies, such as folsom, have the optional 3rd argument “load”, which means “don’t start the application, I just need to call some of It’s functions” (code re-use). All of the others assume the default value and so will be started up when riak is started.

So a partial view of riak’s promise can be visualised like this: 

click to view full size

Starting an application

When the time comes to start an erlang/OTP application the convention is to look in the {application_name}_app.src file which points to a module implementing the application behaviour – akin (loosely) to a class inheriting a base class called “application” in C#, Java etc.

All applications that needed to be started as part of  the promise will also be started in this same way when the main application is started.

The naming convention for these entry-point modules is usually {application_name}_app.erl – rebar will even guide you down this path.

How riak does it

Funnily enough riak has no code; it just ensures all the applications it needs (it’s promise) will be running with some slight configuration. So there's no entry-point to look at.

Cheer up, though, because we can definitely look at the entry-point of the riak_core application instead.

 In riak core’s case, according to the convention, the start point of the application should be riak_core_app.erl. Here’s the first 20 or so lines of that file just to demonstrate its existence:

The supervision tree

Being built for concurrency, and thus fundamentally asynchronous with thousands or even millions of processes at any one time, erlang needs a facility to keep all of this under control. Supervision trees are the answer.

A supervision tree contains two broad types of nodes – supervisors and workers. Put simply, supervisors start and then watch children. When a child dies, the supervisor will restart it using a pre-configured strategy.

A common convention is that each application has a top-level supervisor with the name {application_name}_sup.erl. This supervisor is normally started in the {application_name}_app.erl file discussed previously.

Here’s the snippet from riak_core_app.erl that starts the tree in riak_core:



riak_core’s supervision tree

In a second we'll look at riak_core's supervision tree, but let me just fill you in on the basics of OTP supervisors.

Supervisor is another OTP behaviour, so supervisors are erlang modules that “inherit” this behaviour. Part of the inherited api is the function called start_link(), as we saw being invoked in the previous code example.

In riak_core’s supervisor, the function is implemented like this:


Take my word for it (or learn erlang) that the above code will look for a function inside the same module (riak_core_sup) called init() and invoke it. Inside init() we can see the supervisor starting it's children:


Each line starting with “?CHILD” contains the name of an erlang module to start and supervise, with a flag indicating if it’s a worker or supervisor.

The bottom line says “if any one of these modules dies (has an error it can’t recover from) just restart it.”

Those modules tagged as “supervisor” will look similar to this, with a list of modules they are going to start up and watch. But they might have a different restart strategy, such as: if one dies, restart them all” (one_for_all).

Here’s a partial view of riak_core’s supervision tree:

click to view full size

Using Riak to understand how Erlang applications are packaged and started

You've learned the syntax, had a dabble with OTP, and found the time to create a few toy projects - but now you want to build or take apart a real-world Erlang application.

How is the production version of an application created, and how is it started up? In this post I’ll be digging into the source code of riak to answer these questions.

What we’ll see is that:

1. To generate the deployable package (a release) a makefile wraps up a few calls to rebar 
2. rebar gets dependencies, compiles the app, & uses reltool to create a release complete with all the bits and pieces to run it
3. The application is started using erlexec, that uses boot files from the release 
4. A shell script is used to wrap up most of the above complexity 

By the way, I don't code Erlang for my day job so please keep that in my mind when using this information. Also, feel free to tell me if I'm doing bits wrong.

Create a release with rebar & reltool

Riak’s Makefile contains a “rel” target which is the entry-point for generating a release. Here are the relevant bits:


It’s first telling rebar to look in the rebar.config file for dependencies and to download their binaries (beam files) into the deps folder of this project. The compile command tells rebar to compile the .erl files into .beam files for this project.

Now the interesting bit, the “generate” rebar command will create a release for this application. When that’s done a release is added to the “rel” folder in the root of the project, and will look like this:




Digging into the generated release

How did the release folder get named "riak", and how did all those files end up there when a lot of them look specific to the riak application? For this, rebar uses reltool which relies on instructions & settings in reltool.config (at the bottom of the image above).

At the top of reltool.config we can see the application name and version (see image below). The release is called “riak”- explaining why the directory is called riak (see image above).


At the bottom of the same file we can see some overlays, which (I believe) are specific to rebar, and not acknowledged by reltool:


These overlays are almost self explanatory, except for maybe “template”. Template extends copy by substituting the value of any variables in the file with values from vars.config, as instructed in the line beginning "{overlay_vars".

For example, look at the template instruction with the values “files/riak” and “bin/riak”. If you look at the first image showing the folder structure, you can see a file called "riak" in the /rel/files folder and you can see one inside the release’s bin folder (/rel/riak/bin) which was copied during release generation.

It’s also worth mentioning the “erts-5.9.1” folder - this is an executable version of the erlang runtime, while the lib folder contains all the compiled applications referenced by this project. 

Start the application with erlexec

Now the application has been packaged up with everything it needs to run, how is it going to be started? There’s one directory in the release I haven’t mentioned yet – I’ll use it to answer that question.

If we open up the “releases” directory we’ll see this:



Inside the “releases” directory there is a folder "1.3.0rc2" which matches the version that was specified in the reltool.config file. The files that make up this directory are explained in detail on the erlang website. To quote them:

“When starting Erlang/OTP using the boot script, all applications from the .rel file are automatically loaded and started:”

In this case, "riak.boot" is the boot script.

It feels like we're close to the prize here, we just need something to kick these boot scripts off by the looks of things. Fortunately, we can find exactly what we are looking for in the /rel/riak/bin/riak shell script. It will  execute the following when you call “riak start” or “riak console” from the command line:


…where the $BINDIR is the erlang bin directory (/rel/riak/erts-5.9.1) as per the screenshot below:



That command starts the erlang vm in boot mode (using –boot flag) instructing it to look in the release folder for riak.boot & riak.script. It also tells it to start the vm using the args from the vm.args file (which was another of the overlay commands in reltool.config).

The other two flags just tell it to start in embedded mode (-embedded = no console) and add some directories to the path (-pa [directory path]).

Sanity check with escript

When riak was started using “/rel/riak/bin/riak start” the application should have begun along with all of its dependencies. We just need to confirm that now to complete this tutorial. To do so we'll see how riak uses escript to query theapplication which should now be running.

As part of the release, the erlang runtime bin directory contains a file called escript which you can see in the prevous image (just below erlexec). escript is basically an erlang interpreter – you don’t need to compile the code to run it.

Next to the riak shell script, there is also a riak-admin shell script that uses escript to execute the erlang code in the nodetool file (this also lives in the erlang bin folder). Nodetool is typically defined and executed like this:


So when I run "riak-admin status" it is telling escript to dynamically execute this code from the nodetool file:


...which is making an rpc call to riak_kv_console:status() in the riak_kv application (the arguments given in the shell script). If I run "riak-admin status" then, I would expect to see the output of riak_kv_console:status() for the running riak node:



Sweet :)

Next time I'll be looking into Erlang's supervision tree - again using riak.

Simple steps to great diagrams

Diagramming skills are really important for your team, employer and career was the message I wanted to share previously. As promised in that post, this is the hands-on follow-up guiding you down the path of good diagrams.

Scenario

I’d love to show you an example of some real diagrams that I created for systems that I work on, but that’s not going to go down well. Instead, I picked this job spec from Stack overflow, and decided to design a system loosely based on it.

What we have then is a sports trading platform that needs to meet these functional requirements:

•  Provide a website allowing users to view in-play odds for major sporting events
•  Prove an API allowing business users to share with their customers in-play odds for major sporting events
•  Both website and API need to accept payments (bets)
•  Calculate odds that will make the company the most profit
•  Customisable parameters for business partners that determine how aggressive the odds will be
•  Management application for staff to view details of payments, and manually update events

Additionally, there are some non-functionals that will determine how successful this project is:

Scalability

• High-volumes of traffic and always increasing
• Ahead of major sporting events there will be a massive surge in traffic – the system needs to be able to handle this

Audit

• All transaction data must be kept for 5 years
• Cannot afford to lose data

Performance

• All requests for information must be responded to within 500ms

Availability

• If data is not available the business loses about £3000 a minute

In real-life we would obviously expect a more extensive list of non-functionals.

Context Diagram

From the highest level possible, zoomed out to maximum, context diagrams indicate where the system fits into the bigger picture – which systems does it communicate with, and who are the users?

For the sports trading example, I came up with this:

Click to see full size

What’s prominent and purposeful about this diagram is the way it can keep everyone focused on who they are building the system for, and who they rely on. Every day developers will see the diagrams on the wall and feel excited about building a sports trading platform that’s powering applications for some giant sporting brands.

Perhaps more importantly, this diagram is a powerful tool for collaborating with less technical colleagues.

Consider the case when a friend in customer services receives a surge of complaints about payments taking too long. You can show them this diagram, making it easy for them to report to customers: “Cash-blaster process our payments, and they are experiencing technical problems right now”.

Containers Diagram(s)

Twist your lens a few degrees, because the containers diagram zooms in a level from the context diagram. This one most definitely is technical.

With a containers diagram, the idea is to show major technological choices, and interactions between “containers” (think applications for now).

Click to see full size

I wouldn’t like to build a system without first establishing this information. Nor would I want to modify an existing application without knowing the important details a containers diagram shows me (not usually much choice).

If I’m going to write code, I really want to know why I’m writing it, and where it fits into the overall technical solution. That’s why context and containers diagrams make me very happy.

Non-functionals
A containers diagram is useful for validating your design against many non-functional requirements. Using this containers diagram, it’s easy for us to explain we can cluster our NServiceBus host at peak times to handle the surges in traffic for major sporting events (that we've been asked to accommodate).

Availability of odds & event data is crucial for the business, remember. This containers diagram shows how we plan to (or are already) use clustered MongoDB to address this. It’s not a full-on guarantee, but it can help to validate your ideas (fail fast etc).

Component Diagram(s)

Twist the lens a few more degrees, and now we find ourselves just above the class level - we’re in component diagram land.

They are controversial, and might be too much of an implementation detail, but they’re not quite UML. Components are groups of classes, and can be expressed by just boxes and arrows if you like.

An ideal use of component diagrams is when you are doing DDD (domain driven design). In these scenarios you want a strong coupling between your domain diagrams, terminology (ubiquitous language), and your code.

For the sports trading platform, I can see a lot of value in diagrams like the following. Granted, they will be more volatile than containers or components, but that’s inherent to DDD anyway.

Click to see full size

This diagram has terms like “historic closures”, “market forensics” and “proba-params” which are really important in this domain. In this case we really do want our code to look like this – it’s important if we’re going to keep our domain model – diagrams, terminology and code – consistent.

General Tips

Some of the underlying concepts that make these diagrams so effective are:

• Focus on getting the level of detail correct
• Consider the target audience
• Use colour and notations, but make sure they are explained with a nice key

In my experience, you can take these concepts and apply them to any kinds of software diagram you are making.

Thanks Simon

I should probably thank Simon Brown after I just ripped off a tiny piece of his course and book. I just really find these diagrams immensely useful for all the reasons I mentioned in the last post and more. These are really important skills, and our lives as developers would be a lot better if we acquired and used them.

There’s still tonnes more you can learn from his course and book though, and not just about diagrams. I’m always referring back to the book as I ponder “how would Simon do it?”.

So, cheers Simon - you’ve helped me a lot.

Improve your communication skills with expert diagrams

If you’re a developer who wants to contribute to the big picture – the high-level design of the systems you work on – you need to be able to communicate your ideas clearly for your peers to understand. In my experience, with myself included, developers are inherently lacking in this skill.

Fortunately you don’t need to spend years working on muscle memory and instinctive reflexes. You just need a framework for creating clear diagrams at the appropriate level of detail. Fortunately again, such a thing exists, and the skills are quite easy to come by so I’d love to help you.

In this post I’ll try to convince you that the small efforts of learning and creating diagrams can have a big impact on your team, your company and your career. In the next post I’ll show you how easy it is to create effective diagrams. 

How do you know my diagram skills need to be improved?

Having attended Simon Browns architecture for devs workshop, I failed to provide a set of diagrams clearly expressing the design of a system. Then same was also true of the other engineers and architects in the room.

Just recently I ran a customised version of the diagrams workshop in the offices at 7digital for a few of my colleagues. Again, whilst having lots of fun, I noticed the challenges faced by developers to find the appropriate diagrams at the appropriate level of detail with well communicated notations.

You might be different, but that’s 100% of developers so far (including myself) whose skillset would be much improved with greater ability in this area. 

Diagrams are a fantastic way to share knowledge

A former colleague made me laugh once, “shut up and show me the diagrams” he said. And for my money he’s absolutely right. When you’re talking to me, explaining how a system works, I’m too busy creating a mental image and trying to keep up with you.

Given the option, a lot of times I would much rather you show me a diagram first so we can base our conversation around that. Maybe I’m different to you, though?

Even when talking to non-technical people you can aid communication substantially with diagrams. It’s incredibly easy for us developers to talk in complete tech waffle. Yet when introducing diagrams at the right level of detail, I have seen how greatly-enhanced business-developer communication can be.

Diagrams can even be useful within or between developer teams. Often you’ll see new starters given a code base with another developer speaking a foreign language in their ear about how the system works. Maybe even a few ad-hoc diagrams if you’re lucky.

I’m sure you can relate to this - being the new starter and having difficulty taking in all the new knowledge. I blogged before about how I’ve attempted to alleviate this problem with the use of proper diagrams.

I think diagrams can have an even greater role. I want all development teams within an organisation to have a few key diagrams. This, I believe, would result in more knowledge of how the system as a whole fits together, amongst many other benefits - not least the ability to coordinate teams and move developers between them with greater returns on productivity. 

Diagrams encourage you to think about important decisions earlier and keep an eye on risks

Transforming requirements into a working system almost always involves a lot of significant decisions that can drastically affect the success of a project – Solr or Elastic Search, SQL or NoSQL, message bus or http, distributed architecture or monolithic app.

Some combinations of choices are better suited to your expected business use cases. By having a few diagrams that show these relationships you will often see the potential for pain, and avoid it before you have even started coding. Remembering the cost of change increases the longer time goes by, the potential here is massive.

If you are a fan of “fail fast” diagrams are your friend.

Important, too, is ensuring all requirements are met – functionals and non-functionals (such as scalability, security etc). With good diagrams, you get a visual model of the system you can walk through and apply use-cases against. With a wall of text we rely on mental representations, which I find to be a much poorer tool for collaboration.

Diagrams and DevOps

One of the big buzzwords at the moment is DevOps. If you’re a fan of systems thinking, attempting to optimise the whole rather than just individual parts, then you’re probably loving the DevOps culture

Part of DevOps is bringing developers and ops guys closer together. This involves transitions such as inviting your ops team into the design stage of your project. Sharing and collaborating with proper diagrams could be a massive help here. 

stolen from http://dev2ops.org/2010/02/what-is-devops/ 

Think about this example: you’re a .NET shop and want to run parts of a new system on linux; in the process  introducing chef for configuration management.

Is it best to create a design expressing these features, and share it with the ops team? Or is it best to churn out the code and find out sixth months later that for any number of reasons your ops team can’t allow that, or they had knowledge of a much better alternative? If you rely on talking, it can be hard to express all of the details that can make this difference.

With a design, you can get a rough estimate of the hardware you’ll need. The systems team can tell you if that is sensible, feasible, and how long, if possible, it would take them to acquire and prepare it.

Bring these skills to your team

I’ve now outlined why I think proper diagrams are an important communication tool, not just between developers, but amongst the whole business. Addressing this problem, then, in the next post I’ll share a few of the tricks I’ve learned for creating excellent diagrams.

If you’re keen, you can learn from the man himself by attending Simon’s workshop or reading his book….

… however, if that’s not possible (for whatever reason) I’d love to help your team acquire these skills. I can help you run an architectural diagrams sessions, customised to your situation if needed.

I’m not looking to make money, so there is no charge. It’s just a great opportunity for me to learn from other developers, and help them acquire highly-important skills. Email me at nick [at] ntcoding.co.uk