My personal views, thoughts and opinions.

Thursday, February 06, 2014

SAP HANA Cloud: OpenSocial widgets auto-discovery and operation

Recently we added functionality that enables deployment of OpenSocial widgets and their auto-discovery in SAP HANA Cloud Portal. 

As Wikipedia states:
OpenSocial is a public specification that defines a component hosting environment (container) and a set of common application programming interfaces (APIs) for web-based applications. Initially it was designed for social network applications
You can check the developer guide for OpenSocial widgets in the Portal for more information how to create such widget. 

There is also a short topic on how to package the widget which notes that your spec.xml needs to be:
  • publicly accessible resource
  • named so that it ends with the .spec.xml suffix
  • with name, description and thumbnail that will be shown in Portal Content Catalog

Deploying & running your widget as SAP HANA Cloud application allows you to use the Portal auto-discovery. This means that when your application is started successfully on the HANA Cloud, it will be visible in the Portal Content Catalog, so you can build a site with such widgets.

Soon you should also be able to deploy Cloud Portal sites with the widgets layout you created. Of course we will also be able to do auto-discovery for them as well.

The widgets has to be running and accessible no matter if hosted or not in SAP HANA Cloud. This is needed so the Portal can fetch the spec.xml from the publicly accessible URL of the widget application.

Since widgets and sites are relatively small, they will make small application packages (WAR files). To run several widgets you may need several compute units and you may need to pay for all of them.

However there is a way to combine several small WAR files and run them on a single compute unit as a single application. 

Just copy all WAR files in one directory and use the Command Line deploy command to deploy the whole directory. As the documentation of deploy command states you can use the --source parameter to provide:
a comma-separated list of file locations, pointing to WAR files, or folders containing them
The result will be a single application from SAP HANA Cloud point of view (lifecycle, monitoring, billing), having multiple WAR files running inside.
Please have in mind that all these applications (no matter if widgets or plain Java apps) are running on  the same compute unit and they share the memory, CPU and storage resources of the unit.  

So if you have resource hungry applications, perhaps it is not very wise to save money by increasing the response time (or latency) of your customers.

You can however use this multiple-wars-in-a-single-application approach for development or testing environments, modelled for staged development

So use this for testing, but beware the performance impact for productive scenarios.

Thursday, December 05, 2013

Unbricking DD/OpenWRT routers

The recent news about a new Linux worm that attacks routers made me download and flash the latest version of dd-wrt with the hope that it will have newer versions of the binaries and thus ensure better protection.

Unfortunately after update and following restart the router was rendered useless. Obviously the downloaded firmware was damaged. I had a brick, while only a moment ago this was my home router - the ageing Linksys WRT160NL.

I had to search some sites on my mobile only to find out that in order to unbrick it I may:
  • do factory reset
  • use JTAG and serial cable (or on new machines USB-to-serial cable)
The factory reset did not work. Although the router happily flashed its LAN, WAN and power lights it did not establish connection with my Windows 7 machine, nor had its wireless SSID broadcast. 

So I started investigating the other alternative - uploading new firmware via the serial communication header in the router. The sites mentioned TFTP and then it hit me. I managed to flash Buffalo router a while ago, just by using the built-in boot-loader and TFTP PUT request. It should be possible to do the same since the router seemed to have its lights functioning and therefore at least part of the boot-loader working.

I asked Google and found out in OpenWRT wiki that this should be fairly easy to do. The wiki commanded:
1. Turn off the power to the router and leave it off until the final step.
2. Make sure your computer has a static IP address from 192.168.1.x (eg. 192.168.1.4)
3. Make sure the ethernet cable is plugged into one of router's LAN ports and the other end into computer's ethernet port
3. cd to the folder where you have the image
4. change the name of the new firmware to code.bin , then type :
5. echo -e "binary\nrexmt 1\ntimeout 60\ntrace\nput code.bin\n" | tftp 192.168.1.1
6. plug the power into the router, it should flash.
Well needless to say this didn't worked - I was on Windows. I had the Microsoft TFTP client, that established connection instantly and never looked back to retry. 

Fortunately I had Cygwin installed as well. So I just had to download and install the tftp package. Without router and therefore an internet connection.

I've found the OpenWRT wiki on my iPad using internet via WiFi tethering. So I enabled the USB tethering this time, and used it to update Cygwin and add proper TFTP client to my Windows system. I also downloaded older version of the firmware.

It took me a matter of minutes to try the steps above and to restore my router's firmware.

I even flashed the (hopefully) latest and greatest version of the dd-wrt firmware for 160NL.
Re-downloaded of course.

Thursday, October 31, 2013

SAP HANA Cloud: Application properties. Multiple connections

The latest update of SAP HANA Cloud Platform (we've changed the name again) includes two small but important changes. Copied from the release notes:
  • Deployment with multiple connections:
"You can use the --connections parameter in the deploy command  to specify the number of connections to be used during deployment. Use the parameter to speed up the deployment of application archives bigger than 5 MB in slow networks."
  • Application properties listing
"You can now list the properties of a deployed application using the new command [...] display-application-properties."
Multiple connections

I already provided measurements on the impact of using more than one connection in slow or traffic/connection shaped networks in one of my previous blogs. You can now try this for yourself by using the --connections parameter during deployment.

For number of connections you can use:
  • --connections 1 means you want to disable the feature
  • maximum number of connections --connections 5

Although some networks allow higher number of connections than 5, this rarely pays off, unless you are the only person left in the office on Friday evening with the sole purpose to utilize the whole bandwidth of the company's leased internet line.

Splitting an archive in small chunks has performance penalty of 5-10% over simply transferring it via the line. We wanted to guarantee the split was really needed so we added the 5 MiB entry barrier for the new feature.

To determine the right number of network connections you can:
  1. Start  with the default settings
  2. Increase the number from 2 up to 5 looking at the times for deployment:
    [Thu Oct 31 10:22:53 FET 2013] Deployment started....
    [Thu Oct 31 10:22:58 FET 2013] Deployment finished successfully 
  3. Use the number of connections that provided the fastest deploy time
Application properties listing

This new feature is small, but quite important. Without this neat feature the operators and developers had to write down what they actually used as deploy parameters. There was no way to obtain the settings from the cloud.

Now we finally provided a way to get this info and be able to blame either yourself or a colleague for using a certain setting:

C:\sdk-1.39.13.6\tools>neo display-application-properties samples\deploy_war\trial.properties

SAP HANA Cloud Console Client

Requesting application properties for:
   application: test
   account    : i024099trial
   SDK version: 1.39.13.6
   user       : i024099

Password for your user:
[Thu Oct 31 10:35:24 FET 2013] Requesting application properties...
[Thu Oct 31 10:35:25 FET 2013] Request for application properties finished successfully

   runtime-version   : 1
   minimum-processes : 1
   maximum-processes : 1
   java-version      : 6

Wednesday, September 18, 2013

SAP HANA Cloud: Operator's Guide

The new Operator's Guide is available in the official documentation. The guide targets operators that usually:
  • don't have access to the application source
  • have to update to the latest version
  • maintain the stability and performance of the application

The new Operator's Guide has the following sections at present:

Console Client Reference


Contains information about the use of the console client and will gather all console commands that the SAP HANA Cloud SDK provides.

The new thing here is that finally we managed to provide a common convention of the exit codes that can be used when using the commands for scripting some operations.

Although the commands can still have their own exit codes they are now inside predefined ranges. This means that you may chose to care for a specific exit code, but generally you can just rely on the range for most of the operations.

For example handling exit codes range 40 to 109 guarantees you that you've covered all parameter validation errors. The operation may fail because the archive you are trying to deploy does not exist but you don't need to care about the exact exit code since you are sure it is in the range above.
Documents different aspects on the application configuration that can help the operators achieve improved stability and performance. The covered topics include (for now) how to:
  • update your runtime version
  • enable GZip response compression
  • configure JVM arguments
  • use Java 6 or Java 7
  • scale horizontally your application
  • assign roles
  • configure destinations

The section describes most of the parameters present in the deploy command and gives recommendations and explanation of all the features. Of course it contains security and connectivity information as well (the last topics).

We'll take care to expand it with everything you can configure so keep an eye for more information. 


This section contains information about features needed to ensure the update of your application is as smooth as possible:
  • Update with Zero Downtime
  • Update with downtime
  • Soft shut-down 

I've already described most of the features in my previous blog but since we have more to share I intend on writing a follow up to complete the picture with the support for custom Maintenance Page.

As you can see we have laid the foundation for the new Operator's Guide, but it's by no means complete. You can just check the release notes or have a look at the content every two weeks for new topics.

Monday, August 12, 2013

SAP HANA Cloud: gzip Compression

An easy way to save bandwidth is to use gzip compression and send less data to the client's browser.

Googling gzip provides some good explanations of the gzip compression. The first two results:
The rule of thumb is to turn on the compression for text based content (scripts, book, JSON or XML) since it would benefit from the heavily reduced bandwidth. Images and audio-visual formats would usually require no additional compression.

Following this recommendations up to now SAP HANA Cloud automatically compressed text based responses (MIME types text/html,text/xml,text/plain). 

The problem is that you could not say you don't want this to happen or to add another MIME type to the list of compressed responses. Setting compression threshold was also not supported.

To lift this limitation we introduced the ability to specify that you:
  • require compression or not
  • what MIME types you want to compress
  • what is the threshold that turns on the compression

We added 3 new parameters to the deploy command and you can specify when deploying your application:
  • --compressible-mime-type - comma separated list of MIME types for which compression will be used.Default: 'text/html, text/xml, text/plain'
  • --compression - enables or disables gzip response compression. Acceptable values: 'on', 'off', 'force' or an integer
  • --compression-min-size - responses bigger than this value get compressed and ones smaller than the value are not compressed. Default: 2048 bytes

To deploy your application with gizp compression of javascript you can issue: 
neo deploy myapp.properties --compression on --compressible-mime-type application/javascript

Behind the scene we are using Tomcat gzip compression described in the Apache Tomcat's Configuration Reference.

Wednesday, July 24, 2013

SAP HANA Cloud: Updating productive applications

So far the update of the productive applications was entirely in the hands of the developers. Not necessarily a bad thing, but this required lots of boilerplate code that every application has to embed.

The new bi-weekly update of the HANA Cloud will introduce small feature that however will enable the customers to update their application with reduced or no-downtime at all. 

On HANA Cloud you can have one or more instances of your application and each of these instances is called an application process. Previously the allowed actions were:
  • start of a single application process
  • stop of all application processes at once

This however implies that you can increase the worker application processes (scale up), but you cannot scale down.
 
What's new is that you can finally stop a specific application process running on the HANA Cloud's compute units by specifying its process ID. If you wonder what the heck do I mean here is some glossary-style explanations:
  • compute unit - look at it as a hardware box, you anyway pay for this to get more CPUs and memory
  • application process - the software that runs on top of the hardware - basically the SAP server that in turn hosts your own application code. 
  • process ID - the unique ID associated with every  application process. Used as application process name in commands (as the term suggests).

So with this minimal change from user perspective now you can achieve:
  • scaling up & down
  • ageing
  • rolling update / zero downtime

Let's see what these three things mean...

Scaling your application
 

HANA Cloud provided the ability to scale your application from the very beginning of its existence. 

As I said customers were allowed to start new processes, but didn't have the ability to stop a single process. Now this is fixed and you can easily include the new application process id parameter in the stop command:
neo stop myapplication.properties --application-process-id <id>
The list of process IDs is displayed after you issue start command, or you can use the status command to have the list printed. In both cases you'll get something similar to the output below:

Application processes
  ID                                           Status
  a182761d75b18b6fe17ed4285089d6447ae4ab3c     STARTED
  385b2cacd896c45dd39c8f444774329869282b80     PENDING
The next step would be to copy the ID and use it in the stop command like this:
neo stop myapplication.properties --application-process-id a182761d75b18b6fe17ed4285089d6447ae4ab3c
The above command will stop the first process, leaving you only one application process to handle the incoming user requests.

Ageing

The ageing is a way to deal with applications that have issues with resource consumption. They either get too slow or consume too much memory. 

This may be due to badly written code, the use of 3rd party library that has leaks or whatever other reason you may think of. You may recognize this approach from home routers or other home appliances that have poorly written firmware, suffer from bad hardware design or most often both :)

In HANA Cloud thanks to the process id you can stop the unhealthy application processes and start new fresh ones to replace them

Rolling update or Zero Downtime

The most interesting application of the new process ID is to update your application. 

In general you can update your application in three ways:
  • without your customers notice anything (zero downtime)
  • before your customers notice anything (rolling update)
  • after your customers notice a warning (maintenance page)

The maintenance page approach includes adding a banner, window or in general something flashy to get the customer attention and inform them that from day/hour 1 to day/hour 2 they will not be able to access the application. This however is quite disruptive since you'll be out of business while updating and your customers have to be informed and to (eagerly?!?) expect this.

In most cases customers are quite unhappy with the notice/maintenance approach so you'll want to do the update with one of the next two approaches. They both require that old versions of your application can work together with new versions of the same code and data. If this is not the case then you either have to stick to the maintenance page or redesign your application.

If both old and new versions of your application can work together you may decide to stop/disable the new functionality until all processes are updated. This may be needed to avoid backward incompatible data reaching the database or being sent via some channel. 

This means that customers may still use the application as they used to, but some will eventually notice the new disabled functions until you roll out the new version.

If there are only minor changes (or your application can cope with the changes) you may decide to simply replace all nodes one by one and have a real zero downtime update.

Should I stop or should I start?

The rolling update and zero downtime approaches require that a new process is started before stopping an old one. This in general helps to keep the ability of your application to process a certain amount of requests. Stopping before starting would effectively scale down your application, so I would recommend start before stop.

Of course using the maintenance page approach will in most cases require you to stop the whole application without using process IDs at all.

Killing Me Softly

Before you can stop an application process you'll want to stop all incoming requests to it. We have in the pipeline the disable command to help you do this.
 
The problem most operators would face is how to understand when to stop the application or the process without affecting user sessions or data. 

To check the active sessions, you need configure JMX checks for your application by executing the following command:
neo create-jmx-check --account <your account> --application <application name> --user <e-mail or user> --name "ActiveSessions" -object name <object name of the MBean you want to call> --attribute activeSessions --host <SAP HANA Cloud host>
This check allows you to view the number of active HTTP sessions per application (per Web context, the context is part of the object name). 

An example invocation that checks for context path /demo would look like:
neo create-jmx-check -a myaccount -b demo -u s1234567 -n "ActiveSessions" -O "Catalina:type=Manager,context=/demo,host=localhost" -A activeSessions --host neo.ondemand.com
Currently the HANA Cloud support for custom maintenance page and the disable command are non-existent but are working on this.

Wednesday, July 03, 2013

SAP HANA Cloud: Multiple connections deployment

Recently we found out that some networks use shaping for connections to SAP HANA Cloud. Shaping is not really a surprise but what astonished us was that the speed was limited to 700 KiB per second, making deployment of large archives a problem.

For example we had a case where a 140 MiB archive was uploaded for 1 hour and 30 minutes. This brought back the times when I downloaded Apple //e disks from BBS via 300 kbps modem for 5 hours.

To solve the issue we came back with the idea to use multiple connections and workaround the issue. This required changes in the client  (NEO CLI in SAP HANA Cloud SDK) and the server. 

Once we had the implementation completed we had the following data from our tests:

Slow network


The approach we used reduced the deploy time from ~30 minutes to ~3 minutes. As we can see the network in Vancouver can handle up to 8 connections  and increasing the number of connection does not make sense since the upload time increases.

Average network

In Palo Alto we managed to improve the deployment time from ~7 minutes to ~1 minute. This network allows for a great number of connections and the maximum transfer rates were reached with 30 connections.

Fast network


The network in Bulgaria allows for up to 3 connections. Even in this network we can see that the transfer rate is increased by increasing the number of connections.

Possible problems

Some networks will terminate the connections if a limit is reached or just hold the transfer until the connections number is under some threshold. Currently this will break the deployment.

When / How can I try this?

We will use 2 connections by default but you will be able to use the --connections parameter when deploying and:
  • revert to the old behaviour by specifying 1 connection
  • either stick with the default or increase to the maximum allowed 6 connections
Please keep in mind that we will revert to one connection if your deploy archive is under 5 MiB.

We expect this new feature to appear with the next update of SAP HANA Cloud SDK. To check if it is there just try the --connections parameter :)

Google+ Followers