Sunday, July 03, 2016

Abacus: Enabling contribution

In the late 2015, together with my colleague Georgi Sabev, I started investigating how to generate application usage for Cloud Foundry applications. We needed metering and aggregation of that usage across the Cloud Foundry entities (application, space, organization).

We found the newly created CF-Abacus project to be a perfect fit to what we were looking for:
  • open source
  • part of the Cloud Foundry Foundation
  • running as a set of applications/micro-service
  • supported arbitrary usage 


Cloud Foundry applications

We soon found that although Abacus could handle a wide variety of measures, metrics and formulas, there was nothing in place that can get the applications state from Cloud Foundry and somehow translate it into usage. We had to create our own stuff to do the job.

We went on and started creating the cf-bridge - a component that should do 3 simple things:

Well the idea sounded simple enough. However for both me and Georgi this was the first clash with Node.js. And we both hated JavaScript. I always connected it with the endless struggle to get some code working equally well (or equally bad) on all browsers. And even on Internet Explorer 6.

We started poking around and copy-pasting stuff to get something working.

To our surprise the code in Abacus looked and read well (with the exception of variable names). So we quickly borrowed some code snippets and bundled them to form the scaffold of the the cf-bridge.

Now what? We needed to try it out. After some reading on how to do it, we finally managed to craft a set of commands. Started the app and got a lot of errors. First clash with jslint: "The JavaScript Code Quality Tool"Since there was no easy way around it we simply fixed our code to comply with the project settings.

Finally we had our app ... well not really working, but at least started. We previously worked on extending Diego. And Diego was a TDD project. So the idea of having a test before we continue further was not too far and did come quite easy. The Mocha test framework was also pretty familiar to us. Looked and behaved like Ginkgo or RSpec.


Time-based metrics

Since this is not the end of the blog you might expect we had some more troubles. Next obstacle we faced was the lack of good way to measure usage for Cloud Foundry applications. We could sample the usage, but we did not like this idea much.

Next thing we know if how we write to the Abacus team on Slack. After several long threads in which Jean-Sebastien Delfino tried to explain us how things work, he decided to rid us of our misery and proposed a special kind of metrics that could handle our use case - the "time-based metrics".

The Abacus team even created a sample plan to help us go forward and implement our bridge.


Contribution enablers

We managed to create a working resource provider for metering CF application memory - the cf-bridge. But that wouldn't be that easy (or even possible) without a few things that enabled us to contribute to Abacus:
  • clean project code   
Abacus is an open-source project. Having a clean code is pretty important for contributors. They have to be able to understand what's the intention of a snippet without additional documentation. Or at least without a detailed doc on the subject.
  • code quality tools
What kills your app, teaches you a thing or two. We didn't have much experience with JS and wrote code that looked like machine-translated from Java or Go. JSLint helped us get up to speed on JS syntax. Plus it puts the code conventions on a higher level (almost where golang is). Code quality tools are a must for a project that wants to enable as many contributors as possible.
  • ECMA 6
Abacus used some ECMA 6 features long before the official support in Node.js was available. Most of them are just a "syntax-sugar". But some made our life easier and our code cleaner. All these new and shiny features made us think that we might be wrong about JavaScript.
  • test tooling
Mocha is a great framework, but Abacus added lots more. We found out that the chai and sinon modules were automatically prepared for us. What's more we liked the code coverage reports generated on every test run to help us see what's left to be tested.
  • dev turnaround 
The dev process with Abacus is one of the easiest I've experienced. Partially because we used a dynamic language, partially because of Node.js and npm, but mostly because of the tooling around the project.
  • welcoming community
You may have the perfect tooling, but in the end you are working with people. And if these people are not welcoming, willing to help and encouraging your contribution, you cannot do much. No matter that it was obvious that we are not experienced in the matter, Abacus commiters provided tremendous help. Without them we would not be able to create the code and donate it to the project.
Bottom line
 
Both the tooling and the community helped us to do our job. Without them it would be pretty hard to contribute. Not really a surprise I guess.

But if you don't have one of them you'll need to somehow balance and stake a lot more on the other. For example bad or missing tooling can be compensated by a good community. But what about vice-versa?

Well sometimes you need to change your way of thinking as well. Then only a good community can help. As is the case with Cloud Foundry Dojo program.

id_rsa.pub: invalid format, error in libcrypto

After I upgraded my Linux and got Python 3.10 by default, it turned out that Ansible 2.9 will no longer run and is unsupported together with...