Category Archives: Architecture

Amazon Cloud Certification

AWS is extremely hot right now — according to Ryan Kroonenburg from A Cloud Guru, it is estimated there is a skills shortage of approximately 1.7 million certified AWS professionals worldwide. The cloud is now the uncontested default for startups and for reorganisations of IT infrastructure in general.

Up until quite recently, there used to be five AWS certifications:

AWS certifications, 5

The Associate ones can be approached in any order, but it’s considered a good idea to start with the Solutions Architect Associate as it lays the foundations for almost everything else, then the Developer Associate, and finally the Sysops Administrator Associate certification.

The two Professional ones are considered difficult, the AWS Solution Architect Professional in particular: some people say it’s one of the toughest IT certifications in the field. Both require extensive preparation; practical experience with AWS is of course invaluable.

To progress to the Professional certifications, Associate level certifications are required:

AWS certifications, tiers

To qualify for the Architect Professional, you must be certified as an Architect Associate. The Devops certification requires you have either the Developer Associate or the Sysops Associate.

After working every day in the Amazon cloud since 2012, I thought it might be a good idea to get certification for the kind of work I’ve already been doingEven more so since many of my colleagues at Diabol AB already have AWS certifications and have worked in the cloud for years. 

So, a few days ago I passed the AWS Solution Architect Associate certification.

aws-certified-solutions-architect-associate

My end goal is to take all five. In addition to the Architect Professional, the Devops Professional certification is particularly interesting given that Diabol AB specialises in Continuous Delivery and has deep expertise in Devops and Lean/Agile methodologies.

So, to prepare for the Pro ones, I’m taking all three Associate certifications. Since their contents overlap to a large extent — I’d say they are 70% similar — it makes sense to study them together. If all goes well, I’ll have all three before the end of this month.

After that, I need to concentrate on a few AWS technologies and services which I haven’t used yet — AWS is such a rich environment now that nobody knows the entire platform: you have to specialise — before I go for the DevOps Professional. I don’t know exactly when I’ll sit for that. And finally, after some further preparation, the AWS Solution Architect Professional.

And to anyone who has one of the Associate level certifications, I’d say: get the two other ones. It’s not that much work.

In the coming weeks, I’ll be posting updates covering the various certifications, including thoughts and reflections on the contents and relative difficulties of the respective exams, and also a little on how I’ve studied for them.

AWS Summit recap

This week, the annual AWS Summit took place in sunny Stockholm. This article aims to provide a recap of my impressions from the event.

It was evident that the event had grown from last year, with approximately 2000 people attending this year’s one day event at Waterfront Congress Centre. Only a few session were technical as most of the presentations just gave an overview of the different services and various use cases. I really appreciated the talks from different AWS customers who spoke about their use of AWS technologies and what problems they solved and how. I found it valuable to hear from different companies on how they leverage certain products in their production environments.

The opening keynote was long (2 hours!) and included a lot of sales talk. The main keynote speaker mentioned that 20 percent of the audience had never used any AWS services at all, which explains the thorough walkthrough of the different AWS products. One product which stood out was Amazon Inspector, which can detect and remediate security issues early in your AWS environment. It is not yet available in all regions, but is available in e.g. eu-west-1 (Ireland). It was also interesting to hear about migration of large amounts of data using Snowball, a physical device shipped to your datacenter, which allows you to move your data faster than over the Internet (except for the physical delivery of the device to and from your own datacenter).

It is undeniable that Internet of Things (IoT) is gaining traction and that the amount of connected devices around has grown exponentially the past few years. AWS provides several services for developing and running IoT services. With AWS IoT, your devices can securely communicate with your backend servers. What I found most interesting was the concept of device shadows. A shadow is an interface which allows you to communicate with a device even though it would be offline at the moment. In your application, you can communicate with the shadow without the need to care about whether the device is online or not. If you want to change the state of a device currently offline, you will update the shadow and when the device connects again, it will get the new desired state from the shadow.

At the startup track, we got to hear how Mojang leverages AWS for their Minecraft Realm concept. Instead of letting external parties host their game servers, they decided to go with AWS for Minecraft Realm, to allow for a more flexible infrastructure. An interesting aspect is that they had to develop their own algorithm for scaling out quickly, as in a gaming environment it is not acceptable to wait for five minutes for an auto scaling group to spin up new machines to meet the current demand from users. Instead, they have to use quite large instance types and have new servers on standby to be able to take on new traffic as it arrives. It is not trivial either to terminate instances where there is people playing, even though only a few, that wouldn’t provide a good user experience. Instead, they kindly inform the user that the server will terminate in five minutes and that usually makes the users change server. Not ideal but live migration is too far away at the moment. They still use old EC2 classic instances and they will have to do some heavy lifting to modernise their stack on AWS.

There was also a presentation from QuizUp on how they use infrastructure as code with Terraform to manage their AWS resources. A benefit they get from using Terraform instead of Cloudformation is to get an execution plan before actually applying changes. The drawback is that it is not possible to query Terraform for the current resources and their state directly from AWS.

In the world of relational databases in AWS (RDS), Aurora is an AWS developed database to maximise reliability, scalability and cost-effectiveness. It delivers up to five times the throughput of a standard MySQL running on the same hardware. It is designed to scale and to handle failures. It even provides an SQL extension to simulate failures:
ALTER SYSTEM CRASH [INSTANCE | DISPATCHER | NODE ];
ALTER SYSTEM SIMULATE percentage_of_failure PERCENT
* READ REPLICA FAILURE
* DISK FAILURE
* DISK CONGESTION
FOR INTERVAL quantity [ YEAR | QUARTER | MONTH | WEEK | DAY | HOUR | MINUTE | SECOND ]

Probably the most interesting session of the day was about serverless architecture using AWS Lambda. Lambda allows you to upload snippets of code, functions to AWS which runs them for you. No need to provision servers or think about scalability, AWS does that for you and you only pay for the time your code executes in units of 100 ms. The best thing about this talk was the peek under the hood. AWS leverages Linux containers (not Docker) to isolate the resources of the uploaded functions and to be able to run and scale these quickly. It also offers predictive capacity planning. An interesting part is that you can upload libraries which your code depends on as part of your function, so you could basically run a small microservice just by using Lambda. To deploy your function, you package it in a zip archive and use Cloudformation (specified as type AWS::Lambda::Function). You’re able to run your function inside of your VPC and thus leverage other resources available within your VPC.

All in all I thought this was a great event. If you didn’t attend I really recommend attending the next one – especially if you’re already using AWS.

As we at Diabol are standard partners with Amazon, not only can we assist you with your cloud platform strategies but also to tie that together with the full view of your systems development process. Don’t hesitate to contact us!

You can read more about us at diabol.se.

Tommy Tynjä
@tommysdk

Agile Configuration Management – intermezzo

Why do I need agile configuration management?

The main reason for doing agile configuration management is that it’s a necessary means too achieve agile infrastructure and architecture. When you have an agile architecture it becomes easy to make the correct choice for architecture.

Developers make decisions every day without realizing it. Solving that requirement by adding a little more code to the monolith and a new table in the DB is a choice, even though it may not be perceived as such. When you have an agile architecture, it’s easy to create (and maintain!) a new server or  middleware to adress the requirement in a better way.

What is app oriented architecture?

I have had the opportunity to discuss IT solutions in large governmental institutes and large companies with business people the last couple of days, and I have got to learn a new word: “app oriented architecture”. It is not one or two times, it is more a rule than exception that these people have been deeply influenced by applications on IPhone and Android.

– “We are looking at designing a central data storage where we have all the business logic and data, and then create small applications around that for all the functionality we need. Like this (a mandatory finger pointing at a phone – every time) small simple applications that can be replaced and easily developed by contractors.”

To me it sounds a bit like SOA, but these persons don’t share that view:

– “We had a SOA project a couple of years ago, it was a complete failure, we will do it right this time.”

Ok, being a software developer and architect I had never thought about IPhone applications as being something you should base an IT architecture around. But clarely there is something about the IPhone that makes people think in new ways.

So what is app oriented architecture then? Well when you think about it, the idea is not all bad, in fact it is really, really good. Not in a development/technical way, but in a pedagogical way. Suddenly we have means to talk about functionality and architecture in a way that the non geeks can understand, because suddenly they understand that functionality should be lightweight, contained, based on a common platform, available as a packaged solution on demand. And when I started to think more about that, it is clear that the geeks (developers and architects) can learn from that as well. Less integration, clearer interfaces, user driven deployments, unused apps gets outdated, improved rollout of upgrades, etc… The IT department could be organized as an “appstore” providing inhouse and near house developed functionality packaged as components and deployed on user demand.

But why is it that the business people thinks that this is all better than SOA (and why do I agree)? Well the main problem with SOA is that it is too technical, even the name Service Oriented Architecture talks about a technical solution on the IT side of the problem, not the business side. So SOA takes a technical approach to solve a business problem and at the same time limits the IT department to a heavy rigid platform with ESBs and WebServices.

The paradigm shift where is that it has suddenly been clear to business people what a “service” really is. It is not a WebService interface on the financial system, or a security service in an expensive product or some other technical beast. It is a piece of functionality that is clear to everyone what it does and why – just like “I use this app to look at the weather forecast”. Suddenly business people can talk about business (not weather) and geeks can focus on putting that business into systems. If we have to call it app oriented architecture, then that is fine by me. As long as I’m not forced to use web services or ESBs just because a “SOA provider” need to make money.