My personal views, thoughts and opinions.

Tuesday, November 06, 2018

Downloading pictures from Flickr

Recently Flickr announced that Free accounts are going away as we know it (1TB free storage): https://blog.flickr.net/en/2018/11/01/changing-flickr-free-accounts-1000-photos/

Me and my wife have different accounts. I quickly came to the conclusion that I had to pay twice or merge the accounts. I don't need any of the shiny features Flickr gives for "free" with Pro account. So moving to alternative solution came to mind. But how much space/money it would take to move to a new site?

To find out, I decided to download all my pictures. That turned out to be a hard task, using Flickr only tools/features. Flickr's "Download Album" generates unique id that does not contain album name. When you download more than 2 albums on your disk, it starts to feel like detective work finding which 12345abcd like zip is your "birthday" album.

Well, it turns out Pro has benefits here - you can download photos via the Desktop app. Well there should be other Desktop apps out there.

That's why I turned to a great small app I used before:

Another way is to simply pay for a monthly Pro account and use the Flickr's app.

Friday, April 13, 2018

Installing Windows on external USB hard-drive from Linux

Create VirtualBox machine with Windows:
  • Download Windows
  • Create VM and install Windows
  • Install Extension pack
  • for WinXP you might need to install USB 3.0 driver

In Windows VM:
  1. Download and install Win2USB
  2. Download Windows installation image
  3. Connect the external HDD and enable it for VirtualBox guest
  4. Start Win2USB and select install to the USB HDD

Wednesday, October 04, 2017

Node.js and MongoDB with Cloud Foundry & SAP CP

I recently created a step-by-step guide on how to use Node.js and MongoDB in SAP Cloud Platform. It focuses on Cloud Foundry and Node.js development practices I've gathered developing several applications.

The source of the sample app is available here:
https://github.com/hsiliev/cf-mongo-sample

Monday, February 20, 2017

Apple TimeCapsule from Linux Mint 18

To connect to Apple's TimeCapsule from Linux Mint 18.1 you need to add to /etc/samba/smb.conf global section:
#
# Enable to access Apple TimeCapsule
#
client use spnego = no
Then restart samba with sudo service smbd restart and point your Nemo to //timecapsule_address/Data

Saturday, January 21, 2017

No USB devices in Virtualbox

I use several VMs with Virtualbox for testing and running old but useful programs. I even had my Cannon printer working in the guest OS.

After I updated to the latest Linux Mint last week, the printer was no longer visible.

I did some searching and came across the following thread that helped me solve the issue:

In short I had to:
  • install the correct Extension pack to have USB
  • add my user to virtualbox group
  • log off and back in

Wednesday, November 16, 2016

Abacus: Concourse pipelines

If you work with Cloud Foundry you noticed that most of the development is powered by Concourse CI.

Abacus has 3 Concourse pipelines

Test pipeline

This pipeline builds Abacus and runs tests against Pouch, Couch and Mongo. The last step is to deploy Abacus on Cloud Foundry with in-memory PouchDB as a smoke test.

Future plans: automatic promotion of changes from develop to master branch so we can ensure that master is always stable.

Deploy pipeline

Deploys Abacus large profile and:
  • creates and binds DB service to Abacus applications
  • maps a route to each Abacus collector and reporting instance

The pipeline uses route mapping to get enable monitoring of a specific Abacus instances through CF router.

Abacus configuration is externalized in abacus-config directory that should be provided to the pipeline. The pipeline supports templates to
  • automatically fill the application manifest.yml files
  • extract sensitive information in one central place

Future plans: blue-green deployment.

Monitoring pipeline

Pings all Abacus applications using OPTIONS request. Sets up Grafana dashboard and reports the result in Riemann.

Screenshot:

Future plans: pull more metrics from Abacus.


All pipelines work with custom Docker images we build with the Dockerfiles here.

Friday, November 11, 2016

Abacus: MongoDB support

Abacus initially supported CouchDB. However we wanted to use MongoDB because SAP had existing MongoDB installations and expertise. 

We also wanted to see if we can use the Mongo's aggregation abilities: pipeline and map-reduce operations in the long run.

That was the reason I started porting the existing dbclient module supporting basic CouchDB operations like get, put, remove, allDocs and bulkDocs to Mongo.

Since we already had the specification in the form of unit tests for the couchclient I started implementing the client in semi-TDD manner. It took me a week to understand all the details.

The main difficulties came from the fact that I wanted to keep the CouchDB-based behaviour in the Mongo port. I had to mimic the old behaviour in the new mongoclient module to reduce the impact on the existing Abacus code base. This brought to life two simulated features:

Once unit tests ran fine I started the integration suite. As I expected there were problems. The first one - we wanted to create a DB document with eureka port number as "$". Well Mongo does not like fields starting with $ in documents. Fixed.

The integration tests lead to more simulated features in the Mongo port:


For all of the new features I added unit tests to the existing specification of the dbclient module.

Now I had a working mongo client for Abacus. I hurried to try it on Cloud Foundry installation and to my surprise it didn't work. 

The problem: I was using MongoDB that supported only one DB and I didn't had the permissions to create new ones. I implemented a custom partitioning that used collections instead of databases.

Everything was fine until we needed a bigger and highly-available instance of Mongo. Then we found out we don't support replica sets and in particular in combination with collections. A new test and a new feature.

Time for integration in the project. Jean-Sebastien Delfino came up with the idea to provide a minimalistic "layer" that based on an environment variable selects Couch or Mongo client. This played really well when he combined it with a set of scripts to select the DB in development.

If you want to use Abacus with MongoDB check out our configuration page and the Concourse pipeline (my next blog).