A blog by Ninefold and friends.

Support All The Things!

Does bad support make you mad? It makes us mad! Our friendly support team treats customers the way they would want to be treated.

Since our inception in 2011 we have worked hard ensure our customers have a great time using our product and that if they ever need to reach out to support, they are greeted by a friendly and knowledgeable team.

Our secret sauce is the people we hire, it’s more than just a long list of qualifications, it’s about personality and passion. Our team come from technical customer service backgrounds and each individual is empowered to provide the best service they can with the many tools at their disposal.

Over the past 6 months we have undertaken several projects to help our support team serve you better, these include:

  • Tighter integration between our portal, infrastructure and CRM which means our staff get a fuller picture of a customer during each interaction.

  • Streamlining of the ticketing process including improvements to alerting, escalation and the introduction of ticket priority categorisation.

  • An overhauled service status page for improved incident management (status.ninefold.com)

  • New documentation to assist both new and existing customers (help.ninefold.com)

  • Introduction of NPS (Net Promoter Score) to gauge customer satisfaction and help keep our finger on the pulse.

Over the past 3 months our support team have achieved an NPS score of 63 which is considered world class in the IT/Telecoms space (Go team!) and represents the hard work our team put in each and every day.

We continue listening to our customers and investing in product, UX and process improvements. Make an impact, send us your ideas for improvements to feedback@ninefold.com .

Angelo Livanos - Ninefold Support Team Lead

Server Survey Results

We conducted a brief survey of our current Server customer base to help inform our 2015 product development priorities. We were pleased with the response and some of the information gathered and as promised wanted to share it.

Data/Server Sovereignty - 8.7 (out of 10, average)

The issue of data & server location and sovereignty was clearly the greatest concern our customers have. On both average and rankings, this stood out as a key product feature. Ninefold is already 100% Australian owned and as of the end of May will only have Australian datacentres.

Beyond this, we will continue to work on upgrading our security and ensure all data remains on-shore in line with our current and future work with the Australian Government.

Multiple Availability Regions - 8.3

Our customers identified that having multiple availability regions was a high value feature. We are happy to already provide this, but have looked at some of our tooling as a result and believe we can make it easier to deploy across multiple zones. We will be in touch about this more soon.

Fast Server Deployments - 8.1

As we use our own servers for our internal systems (yes, we dog food), we are also aware of this issue. While we would love to offer an easy fix for this, it is a very complicated area, especially when you consider the security aspects of the servers that we provide. In short, there are many asynchronous tasks.

Having said that, we have raised up continued improvement of server deployment speed and hope to make some announcements on this in the coming months.

Server Monitoring - 7.7

We were very pleased to see this as high on the list as it was as we were already working on monitoring integration, specifically through New Relic. This is something we use ourselves (dog fooding) and have already made strides on this being available. We will continue to have this as a high priority item our our product delivery scheduling.

Docker

We are excited by Docker and what it holds for the server and deployment market. We are moving key internal systems to Docker. Interestingly, the response from the majority of our customers was that Docker was not yet on their roadmaps. Indeed, very few responses said they were actively in the process of migrating to Docker.

We see this as an opportunity to help our customers on their Docker journey when they are ready to embark on it. We will commit to bringing interesting news, features, and examples to our community of the progress being made.

Of course, it validates our commitment to providing our existing feature set and will look to add to our current core offering to make it easier and seamless to hosting Docker applications and deployments going forward.

Ninefold Uptime

A new Ninefold Australian-based customer recently asked about how our uptime compared to competitors.

A few of us hadn’t looked at our APAC uptime for a while but we had a feeling it was good; we couldn’t immediately remember any incidents. This peaked our interest, so we did some analysis thanks to our friends at Cloud Harmony:

logo

What immediately jumped out is Ninefold had the lowest downtime over the last year across both our APAC DCs except for AWS*. We were one of only 4 providers to provide 4 Nines uptime measured over the last year.

Why the asterisk for AWS? AWS claims to have 2 standalone deployments in AU, yet they only show up as a merged uptime measure. We are not sure what this means specifically, but it doesn’t seem to capture their October outage.

So, to answer the question, we compare pretty well and the data proves it. Market leading might be a good term. And a call out to our ops team who deliver this every day.

Of course, we always make this information available through:

Total Cost of Hosting Ownership (TCHO): What Goes Into It?

It costs money to provide online servers to host any app, website, or other product. In most cases, though, analysis of these costs begins and ends with an online calculator, if the cost issue is even raised at all. And because cloud servers are relatively cheap and can easily be spun-up and spun-down on demand, most companies don’t look any farther than that.

Unfortunately, it’s usually not as simple as that. Thinking of hosting or cloud service costs as no more than a monthly line item on your credit card misses many of the true costs of hosting. For a realistic picture, you need to think about the Total Cost of Hosting Ownership, or TCHO.

shutterstock_124697734

Your TCHO is made up of four factors:

  1. The actual cost of hosting. This is the easiest to identify and compare, but may be the smallest component of the total cost.
  2. The performance of the app, which can impact customer experience and ultimately revenue. This depends on how much revenue you generate through your app.
  3. The internal cost of deploying your code into production. This is the opportunity cost of the time it takes to actually get your code into production.
  4. The internal cost of monitoring and managing your production environment. This hidden cost covers making sure the networking is sound, the servers are working, and that customers can actually use your app. You know, ops.

To help explain the four components of TCHO, let’s look at the fictitious ABC Corp., an app-based, online-only, Software-as-a-Service provider earning $2 million yearly revenue. ABC Corp. has a CTO/CEO, five full-time developers, one testing/release manager, a support tech, a COO who looks after finance as well, a marketing manager, a community engagement manager, and a marketing systems admin. Let’s assume the 11 staffers earn an average of $150,000 for a wage bill of $1.65 million a year. The company also pays for a variety of systems (Jira, GitHub/BitBucket, BaseCamp, CRM, CMS, MarComms automation, billing, and more) that adds another $200,000 in costs, and ABC Corp. pays $50,000 in annual rent. That’s a total of $1.9 million in costs leaving $100,000 in profit … before we get to hosting.

1. Actual cost of hosting

The actual cost of hosting is the easiest to quantify, but relying solely on this figure leaves out many important expenses.

The choice of a hosting company is typically made by the developer team or the COO. Developers usually select a vendor they’ve used before, or perhaps a hot new vendor they’ve recently heard about. The COO looks at some calculators online, pretends to listen to the dev team, then chooses what he thinks is cheapest.

Lets visit our friends at ABC Corp., who currently leverage a basic Infrastructure-as-a-Service (IaaS) vendor for hosting–roll your own servers, no bells and whistles. Their hosting requirements run $2,000 a month ($24,000 a year) including testing and staging environments, and all the usage costs. ABC has also tested a Platform-as-a-Service (PaaS) offering that would cost $3,500 a month ($42,000 per year). They went with the IaaS vendor hoping that the $18,000 yearly price difference would drop right to the bottom line as profit.

Did ABC Corp. make the right decision?

2. App performance

The performance of an app can be measured in both page load time as well as the consistency of this performance. Both measurements matter in very different ways. Here’s some of the easiest ways to improve customer conversion and revenue with simple app performance improvements.

Faster page load speeds have long been linked to customer retention and sales. As quoted by CNET, Google’s data suggests a 500ms erosion in load time resulted in a 20 percent reduction in click through. Amazon reportedly provides an even easier metric: for every 100ms improvement in page load, revenue increased 1 percent.

Does 1 percent really matter? Well, for ABC Corp., every 1 percent represents $20,000 in additional annual revenue. If a different hosting provider could improve ABC’s average page load time by 300ms, that could mean $60,000 in additional revenue. If the PaaS offering delivered that improved performance out of the box, its additional cost of $18,000 would be more than offset by the additional revenue, resulting in an extra $42,000 in profit.

Consistency of app performance is even more important. Every interaction each user makes (typically a click) impacts their experience of your app or site. Their experience is not just the first page load, but their entire journey, which involves multiple clicks. This is something that we have tested extensively at Ninefold.

What does that mean for hosting and revenue? Let’s make some assumptions:

  1. 5 percent of clicks—1 in 20—are inconsistent
  2. 80 percent of the affected users will continue blithely along even after a slow page load—a very optimistic estimate
  3. The typical transaction path has 13 clicks.

Why is the number of clicks important? Because 1 in 20 clicks is not the same as 1 in 20 users. To calculate the number of users affected, remember Statistics 101 in college. The chance of each click not being affected is 95 percent. However, each click on the journey is subject to this same risk. Of course, not every click has the same impact, but to simplify the math, let’s assume they’re all equal.

The chart below shows the chance a user will face at least one “inconsistent click” based on the number of clicks in the process:

TCHO chart

With 13 clicks, one of every two visitors will experience the “inconsistent click” phenomenon.

What does this mean for ABC Corp.? If 50 percent of its users experience the inconsistent performance, and of those 20 percent leave and do not transact, the company is forfeiting 10 percent of its transaction funnel. On $2 million in revenue, that works out to $200,000 of potential revenue that could be lost due to inconsistent app performance.

3. Deployment costs

These are the costs for your team to get every release pushed and stable. How much can this cost? ABC Corp. has six people in its dev/release team, for a total of $900,000 in staff costs. The team runs on three-week sprints, with 17 releases a year.

If we assume that a PaaS deployment solution would save an average of half a day for the entire team for each release compared to staying with the IaaS provider. That’s a total of 8.5 workdays per team member, about half a sprint. Looked at another way, it 3.4 percent of the 250 available workdays for the entire year. At our assumed rate of $150,000, that comes out to $30,600 of wages.

4. Monitoring and managing your production environment

Given the importance of performance, it’s clear that you also have to devote resources to deployment, to keeping your app running and your systems flowing, and your networking operating smoothly. That typically means employing a SysAdmin or instituting a DevOps structure.

So how much would a DevOps approach cost? Let’s assume that each dev team member spends one week on rotation monitoring the environment. Further, let’s assume that each week a developer is on DevOps watch she loses 50 percent of her development time if ABC Corp. is using an IaaS solution vs. just 25 percent of her time with a PaaS solution. For ABC Corp., with a $150,000 developer cost, DevOps costs $75,000 in an IaaS world but just $37,500 in a PaaS world.

Adding up the Total Cost of Hosting Ownership

Let’s review the TCHO situation for ABC Corp. It currently spends $24,000 per year for basic Infrastructure-as-a-Service hosting. Going to a more comprehensive Platform-as-a-Service hosting offering would cost $42,000, an $18,000 a year difference.

But according to our calculations and assumptions, the PaaS solution would have the following effects:

  • Improving App Performance would bring in $60,000 in additional revenue
  • Making App Performance more consistent would bring in $200,000 in revenue now being lost
  • Improved deployments procedures frees up $30,600 of developer time
  • Reducing DevOps commitments for developers would release another $37,500 worth of developer time

That’s a total of $328,100 in annual revenue gains and productivity savings. Subtract the $18,000 price differential and you get $310,100 in net benefits for ABC Corp. That’s enough to hire two more developers!

Of course, these figures and assumptions may not directly match your company’s situation, but the principle is clear. Considering only the actual costs of hosting ignores important factors potentially worth far more than any out-of-pocket savings.

Spreadsheet image courtesy Shutterstock.com.

The Vibe

The Vibe: “Northern Holiday edition”

I have been on holiday at the beach. I highly recommend it. I swam a lot. I saw a whale. And, to be honest, I kind-of forgot all about the Vibe. Sorry about that.

Scriptster - Making your Ruby scripts all hipster

Scriptster is a simple boiler plate for using Ruby as a (shell) scripting system.

Proxygen - HTTP framework

Facebook released an interesting C++ HTTP framework for building high performance HTTP servers. An excellent reference for modern C++ in action.

Coordination Avoidance in Database Systems

Summary of a rather deep paper pdf on coordination in db systems

Serializing transactions is sufficient for correctness, but it is not necessary for all operations of all applications. The downside of serialization is that it kills scalability and is overkill in many cases.

Playing with the Elixir and Go concurency models

Using Elixir to model Go’s Channels. An excellent dive into what makes these two languages such solid choices for concurrency.

Microsoft announces Remote IE; run IE on your Mac almost Natively

What it says on the box: run IE on OSX. Very very handy.

DevArcs - On the philosophical ramifications of microservice architectures

Microservices pull software architecture from the high-level and abstract to a concrete and emergent concern of a truly cross-functional development team.

8 Docker development patterns

Here I will outline some patterns that have started to show up repeatedly in my use of Docker. Very useful if you are just starting to learn Docker. Which is so hot right now. As you may have already heard.

Raptor

Raptor is “a radically new Ruby web server”. Which seems only slightly hyperbolic. Appears to have some sort of caching built-in.

Distributed Data Types – Riak 2.0

One of Riak 2.0’s most exciting features is support for Commutative Replicated Data Types or CRDTs, including Counters, Flags, Sets,Registers & Maps.

RefluxJS

RefluxJS is a simple library for unidirectional dataflow architecture inspired by ReactJS Flux.

State machines in Ruby

I’m not a fan of the currently popular Ruby state machine gems (state_machine, AASM). State machines are simple, and these are complicated gems with large APIs.

Internet Arcade

The Internet Arcade is a web-based library of arcade (coin-operated) video games from the 1970s through to the 1990s.

The Hammer-Feather Drop in the world’s biggest vacuum chamber

The world is amazing


The Vibe is an eventually consistent weekly collection of interesting links and … stuff.

It’s Ruby, it’s Rails, it’s programming, it’s databases, it’s the vibe, and, uh … No, that’s it.

It’s the vibe[^vibe]

[^vibe]: Confused? Watch this clip from The Castle and you might understand. Watch the whole film and you definitely will.