SQL Script using a declarative style to simplify our data operation. Although HANA SQL Statement is a powerful language, in some cases, especially complex calculation is needed, we need cache table in memory and operate it, just like internal table did in ABAP.

Table variable in SQL Script is a symbol to describe some SQL statement or other logic. It is not a “real” variable at runtime. SQL script’s execution is not line by line, our script was analyzed by HANA and was treated like one or several combined SQL statement. The consequence is we can do nothing with intermediate data you declared in SQL Script as table variable.

Create local temporary table may solve the problem, we could create some temporary table and using update or insert statement to manipulate data in it. But we need to create procedures with write privilege. Thus could not be invoked by read only procedures and calculation views.

Fortunately, SQL Script has an array concept, it could help us to store table data temporary in memory and manipulate it.

In this article, we use a sap standard table SCARR as example to show how to use ARRAY to handle data in memory.

1.Declare Array for storing data

SQL Scrpit do not has a  "Structure" or "Internal Table" concept, so if we want to save data in a table, we should use some arrays, each represent a field.

DECLARE SCARR_MANDT NVARCHAR(3) ARRAY; --Declare a array of  nvarchar(3)
DECLARE SCARR_CARRID NVARCHAR(2) ARRAY;
DECLARE SCARR_CARRNAME NVARCHAR(10) ARRAY;
DECLARE SCARR_INDEX INTEGER;--Declare an integer for record the pointer of arrays

• You could use any primitive data types of sql scrpit.
• Mention that,for some reason, may be some bugs, if you use DECIMAL type in array and try to read its value will lead to a compile error. So if you want to deal with decimals, use double instead and convert it back to decimal after calculation.
• Following a [TABLENAME]_[FIELDNAME] rule to ensure code is easy to understand.

2.Fill arrays with values from a table variable

After binding a select statement to a table variable, using ARRAY_AGG command to fill column data into array

LT_SPFLI = SELECT MANDT,CARRID,CARRNAME FROM SPFLI;
SCARR_MANDT = ARRAY_AGG(:LT_SPFLI.MANDT ORDER BY MANDT,CARRID);
SCARR_CARRID = ARRAY_AGG(:LT_SPFLI.CARRID ORDER BY MANDT,CARRID);
SCARR_CARNAME = ARRAY_AGG(:LT_SPFLI_CARRNAME ORDER BY MANDT,CARRID);

• Sort data by primary key to ensure each array have the same order, then we could use one index to access fields of same record
• Mention to the use of ":"
• Data types of array element and table column must be same

3.Loop over,get data from or set value to the array

FOR SCARR_INDEX IN 1 .. CARDINALITY(:SCARR_MANDT) DO --Using cardinality statement to get number of elements of an array. Array index starts from 1.
IF :SCARR_CARRID[:SCARR_INDEX] = 'AA' THEN --Get value using :ARRNAME[:INDEX]
SCARR_CARRNAME[:SCARR_INDEX] := 'America Airline'; --Set value using ARRNAME[:INDEX] := VALUE
END IF;
END FOR;
SCARR_INDEX := CARDINALITY(:SCARR_MANDT) + 1; --Add 1 to the index to add new rows
SCARR_MANDT[:SCARR_INDEX] := '200'; --Set value with new index directly, be careful not to override existing values
SCARR_CARRID[:SCARR_INDEX] := 'CA';
SCARR_CARRNAME[:SCARR_INDEX] := 'China Airline';

• Up to now, SQL Script do not support directly pass array value to a procedure or function, use a temp sclar variable to handle the data temproryly.
• Mention to the use of ":",":=" and '='

4.Combine arrays to a table variable

Using UNNEST command to combine arrays to a table variable

var_out = UNNEST(:SCARR_MANDT,:SCARR_CARRID,:SCARR_CARRNAME) AS ("MANDT","CARRID","CONNID");
--Arrays are transfered to columns of a table

5.Summary

Using array, we can deal with complex logic which could not deal with SQL statement without create temprory tables, this feature make SQL Script has the ability to handle nearly all data logic.Except this,we could force hana to execute some logic at a certain time and sequence to improve proformence in some case.Although doing those is not as easy as ABAP, we have a way to deal with the most complex logic and combine those logic to the easy unstanding declaretive SQL Script.

This post is part of an entire series

Hana SPS09 Smart Data Integration - Overview

• Hana SPS09 Smart Data Integration - Adapters
• Hana SPS09 Smart Data Integration - Batch Dataflows
• Hana SPS09 Smart Data Integration - Realtime Table Replication

What is the most performing table layout for querying data in Hana? Certainly not the OLTP data model, where many many joins are required to get all data. The OLTP data model is best suited for entering and changing data as it impacts the least amount column/rows. But in that case, is it really such a wise idea to copy the data 1:1 from the source OLTP system into Hana?

Sure you get better performance with Hana than with your existing database, but even Hana is slowed down by every join. Not much but 20 times a little is still a lot. Hence it might be a good idea to read the source data, transform it an load the data into a Hana optimized data model instead of strictly keeping it as is.

Actually, analyzing the cases when the OLTP model is good enough and when transformation makes sense was an entire exercise I went through in another blog post: Comparing the Data Warehouse approach with CalcViews - Overview and Comparing the Data Warehouse approach with CalcViews - Examples.

The summary of my findings are, Hana is simply amazing, how much it can cope with within sub seconds. But the creativity and the needs of business to query data is often even more amazing. Hence my conclusion is, operational reports can be done back where they belong - the operational system if run in Hana. But for true Business Intelligence - finding new business insights in the data - this is where I would suggest building star schema data models still.

How do you transform the data during the load

In the past the answer was quite simple, you use an ETL tool like Data Services. And the answer is and will be valid still. But for just getting data into Hana such an external ETL tool might have a too large of a footprint. They have their own look and feel, their own repository, their own administration and management, even simple things like data types are different. These tool are meant to read any source and put the data into any target.

Therefore we utilized existing Hana technology and combined it to build an ETL feature natively into Hana.

It comprises of the following components

• CalcEngine as ETL engine
• Hana Smart Data Access to connect to a remote source
• Hana security and the entire Hana management actually
• Hana monitoring
• Hana repository for designtime storage and propagation to production
• A new Hana editor, three in fact as of today: AFM Editor, WebIDE based Replication Editor, Smart Data Preparation - another WebIDE based tool aimed towards Business Users

Since this blog is located under Hana Development, let's focus on the tool targeting us, the IT professionals.

The AFM Editor

The AFM editor is a Hana Studio Modeling editor like any other, so I open that from a Hana repository tree.

With this object being created, you can start drawing dataflow. Pull in source objects, target objects and transforms. I am using the term objects here for a reason, because actually it can be many things. Hana tables, virtual tables, views, calcviews, table types, temporary tables. For the ETL use case it is (virtual) tables mostly.

As usual in this blog series I want to do something special, in this case my sources are the 0MATERIAL_ATTR and 0MATERIAL_TEXT extractors of the SAP ERP system. So I drag and drop these virtual tables into the editor and use them as source together with some transforms and have to add a target table.

Note: This adapter is one I wrote, it is not a shipped one.

The interesting part is the types of transforms available in the calcengine now and hence in the editor.

On the right hand side of the editor is a palette and depending on the features being installed in Hana, it shows different categories. If the smart Data Integration and Smart Data quality feature is installed, there are two categories.

Palette "General"

This contains the basic transforms.

• Data Source: Reader
• Data Sink: Loader
• Data Sink (Template Table): A target table that is created matching its input columns
• Aggregation: Allows to group-by columns and specify aggregation functions for the others
• Filter: Allows to specify filters and complex mappings like substring(col1,4,8)
• Join: All kinds of joins (Note: this is not the existing calcengine join. With this join node you can join n tables at once, specify different join types for each, specify complex join conditions.)
• Sort: To order data
• Union: A Union-all operation
• Procedure: In case you want to invoke a procedure with all transformed data being passed into it as table type parameter

Palette "Data Provisioning"

This contains all Data Integration specific transforms.

• Date Generation: A source that generates a range of date values, one per row
• Row Generation: A source that generates a range of integer values, one per row
• Case Node: To route the data through different paths controlled by conditions, e.g. All region='US' data should go into FilterUS, all region='EMEA' data into the transform FilterEMEA and all other rows into FilterOther.
• Pivot: To take n input rows and merge them into n columns. All other columns stay as is.
• UnPivot: The reverse operation, to take n columns, e.g. Revenue_January, Revenue_February,... and create n rows, e.g. the 12 values are returned in the column Revenue but using 12 rows.
• Lookup: To lookup in another table for the lookup value and take one. A join might find multiples and hence the output would be all combinations - often not desired, hence the lookup.
• Cleanse: To cleanse all data based on reference information, most often used together with the postal address directory of every single country of the world.
• Geocode: To translate addresses into long/lat information and vice versa. Or Point Of Interest relationships and the such.
• Table Comparison: To identify for all rows the difference of the incoming data compared to the target table and what type of difference there is.
• Map Operation: To influence the way read data is handled, e.g. all data read from table1 should be flagged as delete and hence will be deleted in the target table. Also to specify different mappings for differently flagged rows, e.g. all insert rows should have the value of now() in the INSERT_DATE column but not be changed by updates. And update rows should write the value of now() into the LAST_UPDATE column.
• History Preserving: Deals with all the complexity when loading a target table that should retain the history. So instead of updaing the e.g. customer record, a new customer record version should be added, the valid-from and valid-to columns of the old and new version should be updated,... things like that. Makes creating a Slow Changing Dimension Type 2 a piece of cake. Note: Requires a Table Comparison Transform somewhere upstream, else it does not know what changed and what the current values are.

Summary

As you concur hopefully, there is little reason to use an external ETL tool for loading data into Hana, even if transformations are required. Although I have talked about loading the data into Hana just once, this solution does allow to create delta logic as well. But more important, it can deal with realtime data as well!

In this blog I explain how to convert an existing Hana database into an Multi Tenant Database. But before we start a short introduction to multi tenant databases

In the past you had the following choices when installing an SAP Hana Database

• One SAP HANA DBMS, one database, one application and one schema.

• One  SAP HANA DBMS, one database, several applications, several schema's (MCOD)

• More then one SAP HANA DBMS (one DB in each) 1-n applications, 1-n schemas (MCOS)
  
  
• SAP Hana Virtualised

Multitenancy refers to a principle in software architecture where a single instance of the software runs on a server, serving multiple tenants. A tenant is a group of users sharing the same view on a software they use. With a multitenant architecture, a software application is designed to provide every tenant a dedicated share of the instance including its data, configuration, user management, tenant individual functionality and non-functional properties. Multitenancy contrasts with multi-instance architectures where separate software instances operate on behalf of different tenants. (http://en.wikipedia.org/wiki/Multitenancy)

A single database container is called a Tenant Database, you can run multiple tenant databases on one SAP Hana System while still having only one software version for a SAP HANA system. Some advantages are:

• strong separation of data and users
• backup and restore available by tenant database
• resource management by tenant (cpu, memory)

When installing a multi tenant database you have two options:

• Start from scratch with a new SPS09 installation, during the installation you get the option to install a single container or a multi container database node
  
  

• Convert an existing single server to a multi tenant database, please not that the change is permanent and cannot be reversed.

Since the installation from scratch is basically not much different from the past with the exception of the screenshot shown above I will focus on converting an existing database in this Blog. I will add future blogs describing details regarding configuration and maintenance of a multi tenant database which will be the same for an new installation and converted system.

Prerequisites

Before you can actually convert a database you have to apply to the following pre-requisites:

• The Statistics server has been migrated or removed
• Your Hana version is on SPS09 or newer

Migrate Statistics Server

As of SPS07 you can migrate the statistics server from a separate process to be part of the nameserver process. When you did not do this before you have to look at OSS notes "1917938 and 1925684" before executing the steps below. Since only installing a database on SPS09 with the "multiple_container" option enabled will install the database by default with the new Statistics server, you will be running the 'old' statistics server unless you manually migrated it already before.

First you need to check whether you are still using the old statistics server, the easiest way to do so is from the operating system with the sapcontrol command to check the processes (sapcontrol -nr <instance number> -function GetProcessList). When you see a separate process for the statistics server (as shown below) then you are still working with the old statistics server and need to migrate it first.

Migrating the statistics server is a simple process, just open the hana studio and go to the configuration. From there adjust "nameserver.ini -> statisticsserver -> activate=true".

After some minutes, you will see that the statistics server is shutting down and eventually it will be removed from the list.

Tip: when you add "watch -n 1" in front of the sapcontrol command you don't have to repeat the command each time manually to refresh.

Finally you can run the SQL Statement “SELECT * FROM _SYS_STATISTICS.STATISTICS_PROPERTIES where key = ‘internal.installation.state'” to check if the migration went fine

Check for the correct hana version

A second prerequisite you have to check is the correct hana version, you most probably already know on which version you run but you can also check it with the steps below (as provided by the admin guide):

First you need to stop the database with the command "HDB stop" and then you execute the command "hdbnsutil -exportTopology <file_name>". When you run the command with the database still running you will get an error message:

checking for inactive nameserver ...

nameserver phyhandb4-prd:30001 is still active. cannot open its persistence failed

Now open the exported file in the VI editor and look for the line "topologyVersion" and check that the value is 15 or higher.

Convert Database

Now that the prerequisites are met we can convert the database to an multi tenant database. In our test we used an empty database, to show you that the existing content is indeed still available after you convert de database we created and empty schema called “CONTENT-MIGRATION” through the SQL command: “createschema "CONTENT-MIGRATION" OWNED BYSYSTEM;”

Before you can convert the database you have to stop the database with the command ”HDB stop”, then run the command “hdbnsutil -convertToMultiDB” to do the actual conversion.

This only takes a few seconds (for an empty database) after which the database is converted. This action executes the following steps:

• Set “multidb mode” flag in the configuration
• Create the system and tenant database
• Updates the secure store in the file (SSFS) system.

The newly created database maintains the original data and has the original port configuration, the initial "HDB start" command only starts the system database though. Therefore it shows that you can start the tenant database by an SQL statement, after you do this it will automatically start when using the "HDB start" command in the future.

However, when you try to connect with hdbsql to this new SystemDB database it will fail with the error:

* 10: invalid username or password SQLSTATE: 28000

This is because you need to reset the password of the SYSTEM user for the SystemDB database. So make sure the database is still stopped (otherwise stop it with "HDB stop") and then reset the system password of the user “SYSTEM” with the command “hdbnameserver -resetUserSystem”. When asked type a new password and press enter.

When the System password has been reset you have to start the database with “HDB start” which will only start the System Database. You can see this with the sapcontrol command, this does not show an additional index server for the tenant database.

Now connect to the database with the hdbsql command “hdbsql -i <instance number> -n <hostname>:3xx13 -d SystemDB -u SYSTEM” after which the system will ask you for a password. You can see that you are connected when the prompt shows “hdbsql SystemDB=>”.

Now you have to start the tenant database with the SQL command “ALTER SYSTEM START DATABASE <SID>”. But when you do this, the system will complain that the user is forced to change its password:

* 414: user is forced to change password: alter password required for user SYSTEM SQLSTATE: HY000

to solve this run the SQL command “ALTER USER SYSTEM PASSWORD <new_password>”. Now run the SQL command "“ALTER SYSTEM START DATABASE <SID>” again and it should work (takes a few seconds before you get your prompt back).

After doing this you can exit the hdbsql command and check if the tenant database is running with the “sapcontrol -nr <instance> -function GetProcessList” command, it should now show an additional XS engine and index server instance “e.g. indexserver-TST and xsengine-TST, where our SID is TST”.

Add a Multi Tenant Database to the SAP Hana Studio

In order to add a multi tenant database to the hana studio, you first have to install Hana studio version “Version: 2.0.7”. After that add a system as usual (right click and choose the option “Add System”).

When adding a “normal” system with the new studio you choose the option “Single Container” but when adding a multi tenant database you choose the option “Multiple Containers”. Each Multi Tenant database has one System Database and one or more Tenant Databases. First we will add the System Database to the studio.

The system user password of the SystemDB database is the one that you have reset in the previous steps.

As you can see, the SystemDB database looks as a normal database but it is only used to maintain the tenant database and not to hold corporate data. As shown in the database above the SystemDB does not contain the schema called “CONTENT-MIGRATION” which we created earlier.

When you choose to install a new SPS09 Database from scratch with the option “Multiple_Containers” then you will have only a SystemDB after the installation and you have to add the tenant databases manually after the installation.

After the SystemDB is added to the studio you can also add the tenant database. The process is the same, but now you choose the option “Tenant database container” and you provide the tenant database name. The System user password of this database is the same as the original database before converting it to a multi tenant database.

As you can see this tenant database does contains the schema called “CONTENT-MIGRATION” which we created before we converted to a multi tenant database.

Now you have converted a single database to a multi tenant database, in future blogs I will write more details about adding and dropping a tenant database and also provide more information regarding the configuration of a multi tenant database (e.g. XS engine access).

Note: Issues facing during ECC-SLT-HANA scenario. In my scenario, each system is standalone system. We have connectivity between three systems.

I have discussed the issues only at ECC system side which we get and highly important.

1. First and primary note is DB USER NAME; it should not be a “SYSTEM” standard user. SAP strongly recommends not using “SYSTEM” as a user for Non-Administrative tasks. i.e. instance, to replicate data to specific SCHEMA and read it from ECC by means of HANA as a secondary database.

For instance, during creating a connectivity between ECC and HANA, HANA as a secondary database, we should not use SYSTEM as username as shown given below.

This screen shot has been captured from "DBCO" transaction code in ECC system.

2. User name and schema name should be same when we are working HANA as secondary database.

For instance, HANA DB Schema name is“ECC_HANA_800” and then DB user name at ECC side should be “ECC_HANA_800”. Because when we are executing the HANA transactions at ECC system side, it would search for tables, views and corresponding HANA objects from schema name similar to our DB user name at ECC side i.e. ECC_HANA_800.

Please create the technical user name seperately at ECC system similar to SCHEMA NAME as shown given below.

This screen shot has been captured from "DBCOCKPIT" transaction code in ECC system.

Please find the screen shot below for reference on HANA DB. Schema Name is “ECC_HANA_800” which is similar to Connection user name at ECC system.

3. We have to configure the connection information as per given below instructions.

For instance please find configuration details below.

HANA Server URL: 172.25.36.125

Port Number: 30015

Connection information would be 172.25.36.125:30015

This screen shot has been captured from "DBCOCKPIT" transaction code in ECC system.

4. We need to have authorization to read the tables, views and respective objects from HANA Schema when we are executing HANA related transactions at ECC system via secondary database access to HANA DB.

For instance, I am executing “FBL3H” transaction code from ECC which would run on secondary database which is on HANA. During the execution of FBL3H transaction code, it would read the data from HANA secondary database.

Since it has no authorization, it would turn into dump at ECC system.

Thank you very much for referring to my blog. I have discussed only ECC system issues.

I will publish my next blog on SLT and HANA system issues for these kind of scenarios.

Regards,

Ravi K Chandragiri

Hi,

I have posted an IDEA on SAP Idea Incubator, about Tracking the Container through the Geo Mapping.

https://ideas.sap.com/ct/c.bix?c=572CDF18-FE78-44D9-9DE4-DA6B17CD5D0C

The basic Concept of this idea is, most of the Business having their own Product and this Products are moving across world based on the customer requirement.

To track this container, through the Geo map, a User or Administrator department could able to very much updated with the any of the shipment.

And even a different kind of Analysis can be done over the any one of the Container / Shipment,

like,

1     Information of the Shipment

2     How frequently, customer is asking the product

3     What is the Current Status of the Shipment and how much time it could take to reach and other shipment related reports etc.

https://ideas.sap.com/SAPHANAIdeaIncubator/container-tracking-system-based-on-the-g

Please help me with your views. Your Suggestion and Feedback would be very much valuable.

Regards.

Praveer.

TechEd Las Vegas in 2013 I had the pleasure of teaming up with Greg Myers from EV Technologies . Quantum Fit was our solution to the explosion of fitness devices without a unified platform. To date, fragmentation of the data is still an issue. This blog will cover the devices that were strapped to me that were posting data to and receiving notifications from HANA.

The application collected biometric, distance and environmental data. This was all posted to an HANA database via XSJS. The HANA application would react to changes in the data to trigger notifications to the athlete. The data was also consumed via an athletes dashboard showing real time data for the athlete.

The demo shows our working solution, after watching the 6 minutes of geeky entertainment I'll go over how we built the device side of the solution.

Smart phone

The center of our Quanitified Self was the smart phone. In this instance we used the iPhone 5S. The overriding purpose was to read the data from the connected devices, device itself, push alerts to the Pebble and post device readings to our HANA System running on AWS.

Location and motion readings were read from the device.

The secondary purpose of the iPhone was to display the current heart rate.

iOS has the CoreMotion Framework that provides access to the Step Count. We enabled both distance determination for outdoor with CoreLocation, but for indoor on stage CoreLocation would not work. In hindsight the step count did not work so well on stage either as there was no guaranteed refresh period from the M7 chip and it slipped outside the time window for the demo.

Accessing the CoreMotion data is relatively straightforward as shown in the code example below.

#include <CoreMotion/CMStepCounter.h>

...

#pragma mark - Core Motion

-(void) startStepCount {

if ([CMStepCounter isStepCountingAvailable] == YES) {

        CMStepCounter *stepCounter = [[CMStepCounteralloc] init];

        [stepCounter startStepCountingUpdatesToQueue:[NSOperationQueuecurrentQueue] updateOn:20withHandler:^(NSInteger numberOfSteps, NSDate *timestamp, NSError *error) {

steps = numberOfSteps;  // Double check on this step count value

self.stepCount.text = [NSStringstringWithFormat:@"%d steps",steps];

[selfupdatePace];

        } ];

    }

}

Pebble Watch

The Pebble Watch at the time was not being used (nor enabled) as a step counter. We looked at creating our own Pebble application but it made no sense as the Pebble comes with a default sports application that the mobile application can use. The following coding is from the original iPhone application. Since this was written Pebble have made a significant SDK update so this may no longer work with the latest SDK.

/*

*  PBPebbleCentral delegate methods

*/

- (void)pebbleCentral:(PBPebbleCentral*)central watchDidConnect:(PBWatch*)watch isNew:(BOOL)isNew {

    [selfsetTargetWatch:watch];

}

- (void)pebbleCentral:(PBPebbleCentral*)central watchDidDisconnect:(PBWatch*)watch {

[[[UIAlertView alloc] initWithTitle:@"Disconnected!" message:[watch name] delegate:nil cancelButtonTitle:@"OK" otherButtonTitles:nil] show];

    if (_targetWatch == watch || [watch isEqual:_targetWatch]) {

        [self setTargetWatch:nil];

    }

}

- (void)setTargetWatch:(PBWatch*)watch {

    _targetWatch = watch;

[watch  sportsGetIsSupported:^(PBWatch *watch, BOOL isAppMessagesSupported) {

[watch appMessagesSetUUID:PBSportsUUID];

        [watch sportsAppLaunch:^(PBWatch *watch, NSError *error) {

NSString *message = @"";

message = error ? [error localizedDescription] : @"Sports App Launch Request";

        }];

    }];

}

-(void) sendDataToWatch {

    NSTimeInterval minsPerMile = cMetersInMile / (paceMetersPerSecond * 60);

    NSString *minsPerMileString = [NSStringstringWithFormat:@"%0.2f",minsPerMile];

NSDictionary *update = [[NSDictionary alloc] initWithObjectsAndKeys:[NSString stringWithFormat:@"%d",self.heartRate],PBSportsTimeKey,

  [NSStringstringWithFormat:@"%0.1f",distanceInMeters/cMetersInMile],PBSportsDistanceKey,minsPerMileString,PBSportsDataKey ,nil];

PBSportsUpdate *upd = [[PBSportsUpdate alloc] init];

    upd.time = 3.95f;

    [_targetWatch sportsAppUpdate:update onSent:^(PBWatch *watch, NSError *error) {

        NSString *message = @"";

        if (error) {

message = [error localizedDescription];

        }

    }];

}

The other feature we used on the Pebble was notifications. This is default behaviour for the watch and required no extra development on our part. The notifications received by the phone are mirrored on the watch.

Smart Water Bottle

The water bottle is where I had most of the fun. We used an Arduino UNO, Redbear Bluetooth BLEMini module and DS18B20 temperature sensor. We had other peripherals such as humidity readers connected but chose not to use them in the demo as we already had a lot to show in 6 minutes.

The coding for the Arduino below and still works when tested yesterday. The temperatue monitor is an I2C serial device which needs the OneWire library for reading data. There are a lot of examples on how to use this, it is one of the many getting started devices.

The RedBear BLEMini was used for the Bluetooth support. Using the serial port the temperature readings were written through the device.

#include <Arduino.h>

#include <SoftwareSerial.h>

#include <OneWire.h>

SoftwareSerial BLEMini(0, 1);

byte blink;

//Temperature chip i/o

/* OneWire */

int DS18S20_Pin = 2; //DS18S20 Signal pin on digital 2

OneWire ds(DS18S20_Pin); // on digital pin 2

void setup()

{

  BLEMini.begin(57600);

  pinMode(12, OUTPUT); 

  Serial.begin(57600);

}

unsigned char buf[16] = {0};

unsigned char len = 0;

void loop()

{

  float temp = getTemp();

  char tempData[10];

  sprintf(tempData,"%2.2f",temp);

  printDouble(temp,100);

  blink ^= 0x01;

  digitalWrite(12, blink);

  delay(1000);

}

void printDouble( double val, unsigned int precision){

// prints val with number of decimal places determine by precision

// NOTE: precision is 1 followed by the number of zeros for the desired number of decimial places

// example: printDouble( 3.1415, 100); // prints 3.14 (two decimal places)

  Serial.print (int(val));  //prints the int part

  Serial.print("."); // print the decimal point

  unsigned int frac;

  if(val >= 0)

      frac = (val - int(val)) * precision;

  else

      frac = (int(val)- val ) * precision;

  Serial.println(frac,DEC) ;

}

float getTemp(){

  //returns the temperature from one DS18S20 in DEG Celsius

  byte data[12];

  byte addr[8];

  if ( !ds.search(addr)) {

   //no more sensors on chain, reset search

   ds.reset_search();

   return -1000;

  }

  if ( OneWire::crc8( addr, 7) != addr[7]) {

   Serial.println("CRC is not valid!");

   return -1000;

  }

  if ( addr[0] != 0x10 && addr[0] != 0x28) {

   Serial.print("Device is not recognized");

   return -1000;

  }

  ds.reset();

  ds.select(addr);

  ds.write(0x44,1); // start conversion, with parasite power on at the end

  byte present = ds.reset();

  ds.select(addr); 

  ds.write(0xBE); // Read Scratchpad

  for (int i = 0; i < 9; i++) { // we need 9 bytes

    data[i] = ds.read();

  }

  ds.reset_search();

  byte MSB = data[1];

  byte LSB = data[0];

  float tempRead = ((MSB << 8) | LSB); //using two's compliment

  float TemperatureSum = tempRead / 16;

  // Convert to f

  TemperatureSum = (TemperatureSum * 1.8 ) +32;

  return TemperatureSum;

}

Notifications

For the demo we chose to use SMS. The quickest way to implement this was to use Twilio the following code is as simple as it gets to send an SMS message. The PHP script was hosted on an external server and called via events from HANA when specific thresholds were crossed in the biometric data.

<?php

// Include the PHP Twilio library. You need to download the library from

// twilio.com/docs/libraries, and move it into the folder containing this

// file.

require "Services/Twilio.php";

// Set our AccountSid and AuthToken from twilio.com/user/account

$AccountSid = "abcdefghijklmnopqrstuvw";

$AuthToken = " abcdefghijklmnopqrstuvw ";

// Instantiate a new Twilio Rest Client

    $client=new Services_Twilio($AccountSid, $AuthToken);

/* Your Twilio Number or Outgoing Caller ID */

$from = '+1 xxx-xxx-xxxx';

// make an associative array of server admins. Feel free to change/add your

// own phone number and name here.

    $people=array(

"xxxxxxxxxx" => "Johnny",

    );

    foreach ($people as $to => $name) {

// Send a new outgoing SMS

$body = "Missing data $name";

        if(isset($_GET["text"])){

          $body=$_GET["text"];

        }

        $client->account->sms_messages->create($from, $to, $body);

echo "Sent message to $name: $body";

    }

?>

Adidas HRM

The Heart Rate Monitor was simply one that supported Bluetooth notifications to allow us to get regular readings. The iPhone app subscribed to receive notifications using the CoreBluetooth Framework. The following code is an extract from the application. Subscribing and listening to the bluetooth devices requires that the devices are found, connected to , characteristics determined and the data read.

#define cAdidasHRMUUID @"0BC904FB-D617-A18E-A6B7-9385378F0A2E"

...

/*

Request CBCentralManager to scan for heart rate peripherals using service UUID 0x180D

*/

- (void) startCBScan {

// https://developer.bluetooth.org/gatt/services/Pages/ServicesHome.aspx

[cm scanForPeripheralsWithServices:[NSArray arrayWithObjects:[CBUUID UUIDWithString:cBiscuitServiceUUID],nil] options:nil];

}

/**

* Invoked when the central discovers a peripheral while scanning.

*/

- (void) centralManager:(CBCentralManager *)central didDiscoverPeripheral:(CBPeripheral *)aPeripheral advertisementData:(NSDictionary *)advertisementData RSSI:(NSNumber *)RSSI

{

/* Make sure we don't pick up a rogue device */

    NSString *cfuuidString = (NSString*)CFBridgingRelease(CFUUIDCreateString(kCFAllocatorDefault, [aPeripheral UUID]));

    if (![knownDevicescontainsObject:cfuuidString]) {

        return;

    }

    NSMutableArray *peripherals = [selfmutableArrayValueForKey:@"heartRateMonitors"];

    if( ![self.heartRateMonitorscontainsObject:aPeripheral] ) {

        [peripherals addObject:aPeripheral];

    }

// Wait until we have all expected peripherals

    if ([peripherals count] == [knownDevicescount]) {

NSMutableArray *uuids = [[NSMutableArray alloc] initWithCapacity:5];

        for (CBPeripheral *per in peripherals)

        {

[uuids addObject:per.UUID];

        }

[cm retrievePeripherals:uuids];

    }

}

/*

Invoked when the central manager retrieves the list of known peripherals.

Automatically connect to first known peripheral

*/

- (void)centralManager:(CBCentralManager *)central didRetrievePeripherals:(NSArray *)peripherals

{

    [selfstopScan];

/* If there are any known devices, automatically connect to it.*/

    for (CBPeripheral *per in peripherals)

    {

[cm connectPeripheral:per options:[NSDictionary dictionaryWithObject:[NSNumber numberWithBool:YES] forKey:CBConnectPeripheralOptionNotifyOnDisconnectionKey]];

    }

}

/*

Invoked whenever a connection is succesfully created with the peripheral.

Discover available services on the peripheral

*/

- (void) centralManager:(CBCentralManager *)central didConnectPeripheral:(CBPeripheral *)aPeripheral

{

    [aPeripheral setDelegate:self];

    [aPeripheral discoverServices:nil];

}

/*

Invoked whenever an existing connection with the peripheral is torn down.

Reset local variables

*/

- (void)centralManager:(CBCentralManager *)central didDisconnectPeripheral:(CBPeripheral *)aPeripheral error:(NSError *)error

{

    if( aPeripheral )

    {

        [aPeripheral setDelegate:nil];

        aPeripheral = nil;

    }

}

/*

Invoked whenever the central manager fails to create a connection with the peripheral.

*/

- (void)centralManager:(CBCentralManager *)central didFailToConnectPeripheral:(CBPeripheral *)aPeripheral error:(NSError *)error

{

[self addLogEntry:[NSString stringWithFormat:@"Fail to connect to peripheral: %@ with error = %@", aPeripheral, [error localizedDescription]]];

    if( aPeripheral )

    {

        [aPeripheral setDelegate:nil];

        aPeripheral = nil;

    }

}

#pragma mark - CBPeripheral delegate methods

/*

Invoked upon completion of a -[discoverServices:] request.

Discover available characteristics on interested services

*/

- (void) peripheral:(CBPeripheral *)aPeripheral didDiscoverServices:(NSError *)error

{

    for (CBService *aService in aPeripheral.services)

    {

/* Heart Rate Service */

        if ([aService.UUIDisEqual:[CBUUIDUUIDWithString:@"180D"]] || [aService.UUIDisEqual:[CBUUIDUUIDWithString:@"180d"]])

        {

[aPeripheral discoverCharacteristics:nilforService:aService];

        }

/* Device Information Service */

        if ([aService.UUIDisEqual:[CBUUIDUUIDWithString:cBiscuitServiceUUID]])

        {

[aPeripheral discoverCharacteristics:nilforService:aService];

        }

    }

}

/*

Invoked upon completion of a -[discoverCharacteristics:forService:] request.

Perform appropriate operations on interested characteristics

*/

- (void) peripheral:(CBPeripheral *)aPeripheral didDiscoverCharacteristicsForService:(CBService *)service error:(NSError *)error

{

// Temperature measurement

    if ([service.UUIDisEqual:[CBUUIDUUIDWithString:cBiscuitServiceUUID]])

    {

for (CBCharacteristic *aChar in service.characteristics)

        {

// https://developer.bluetooth.org/gatt/characteristics/Pages/CharacteristicsHome.aspx

/* Set notification on heart rate measurement */

CBUUID *uuid = [CBUUIDUUIDWithString:cBiscuitCharacteristic];

if ([aChar.UUIDisEqual:uuid])

{

[aPeripheral setNotifyValue:YESforCharacteristic:aChar];

}

        }

    }

// Heart Rate Service

    if ([service.UUIDisEqual:[CBUUIDUUIDWithString:@"180D"]])

    {

for (CBCharacteristic *aChar in service.characteristics)

        {

// https://developer.bluetooth.org/gatt/characteristics/Pages/CharacteristicsHome.aspx

/* Set notification on heart rate measurement */

if ([aChar.UUIDisEqual:[CBUUIDUUIDWithString:@"2A37"]])

{

[aPeripheral setNotifyValue:YESforCharacteristic:aChar];

}

/* Read body sensor location */

if ([aChar.UUIDisEqual:[CBUUIDUUIDWithString:@"2A38"]])

{

[aPeripheral readValueForCharacteristic:aChar];

}

/* Write heart rate control point */

if ([aChar.UUIDisEqual:[CBUUIDUUIDWithString:@"2A39"]])

{

uint8_t val = 1;

NSData* valData = [NSDatadataWithBytesvoid*)&val length:sizeof(val)];

[aPeripheral writeValue:valData forCharacteristic:aChar type:CBCharacteristicWriteWithResponse];

}

        }

    }

if ( [service.UUID isEqual:[CBUUID UUIDWithString:CBUUIDGenericAccessProfileString]] )

    {

for (CBCharacteristic *aChar in service.characteristics)

        {

/* Read device name */

if ([aChar.UUID isEqual:[CBUUID UUIDWithString:CBUUIDDeviceNameString]])

{

[aPeripheral readValueForCharacteristic:aChar];

}

        }

    }

    if ([service.UUIDisEqual:[CBUUIDUUIDWithString:@"180A"]])

    {

for (CBCharacteristic *aChar in service.characteristics)

        {

/* Read manufacturer name */

if ([aChar.UUIDisEqual:[CBUUIDUUIDWithString:@"2A29"]])

{

[aPeripheral readValueForCharacteristic:aChar];

}

        }

    }

}

/*

* Invoked upon completion of a -[readValueForCharacteristic:] request or on the reception of a notification/indication.

*/

- (void) peripheral:(CBPeripheral *)aPeripheral didUpdateValueForCharacteristic:(CBCharacteristic *)characteristic error:(NSError *)error

{

/* Updated value for heart rate measurement received */

    if ([characteristic.UUIDisEqual:[CBUUIDUUIDWithString:@"2A37"]])

    {

        if( (characteristic.value) || !error )

        {

/* Update UI with heart rate data */

[selfupdateWithHRMData:characteristic.value];

        }

    }

/* Updated value for heart rate measurement received */

    if ([characteristic.UUIDisEqual:[CBUUIDUUIDWithString:cBiscuitCharacteristic]])

    {

        if( (characteristic.value) || !error )

        {

/* Update UI with heart rate data */

NSString *temp = [[NSStringalloc] initWithData:characteristic.valueencoding:NSUTF8StringEncoding];

temperature = [temp doubleValue];

self.tempLabel.text = temp;

        }

    }

    }

/* Value for device Name received */

else if ([characteristic.UUID isEqual:[CBUUID UUIDWithString:CBUUIDDeviceNameString]])

    {

        NSString * deviceName = [[NSStringalloc] initWithData:characteristic.valueencoding:NSUTF8StringEncoding];

    }

/* Value for manufacturer name received */

    elseif ([characteristic.UUIDisEqual:[CBUUIDUUIDWithString:@"2A29"]])

    {

self.manufacturer = [[NSString alloc] initWithData:characteristic.value encoding:NSUTF8StringEncoding];

    }

}

Lessons Learned

While everything works in development the real world is a less forgiving place. I had spent the evenings (after midnight) running the streets so that my teammate Clint Vosloo could fine tune the HANA scripts. Timezone differences between the East coast of the USA and South Africa were coming into play here. After midnight there are few people running around wearing fitness devices that could interfere, however the same was not true at TechEd. Sitting in the Monday evening keynote I noticed quirks with the app, it would not always connect to my devices. Pulling out the Light Blue blue tooth testing app showed our issue. There were over 100 wearables in the audience including other HRMs and my coding was picking them all up, I had not added the specific device registration I needed. Good that we found this the night before the demo and not in the demo.

If you ever plan to have a demo that is tuned to your heart rate, make sure you take into account your own stage presence. During our many hours (I ran a lot) of tuning we collected lots of profile data on my heart rate. We tuned the alerts to the expected profile. What we had not accounted for was that my heart rate was already over 20bpm more than my usual resting rate when we started, nervous on stage perhaps? This meant that the system was detecting that I was working much harder than I physically was. While the metric was still correct in that my HR was in an abnormal zone it definitely had an affect on the timing of the demo.

Many thanks to Greg Myers, Clint Vosloo and Eric Vallo for teaming up with me on a fun time.

Firstly thanks to my customer for the situation that forced me to explore this scenario.

I was unable to find a simple document on this explaining such a setup. Hence this blog.

The scenario is to access HANA tables from within Oracle via database links.

Source database : SAP HANA SP 08 Rev 85

Target database : Oracle 11.2.0.3 running on RHEL 6.x X64

In middle Oracle gateway running on RHEL 6.x X64

Target oracle database will migrated to HANA in 6 months, but in the interim period we need to access HANA data sitting inside oracle seamlessly to refresh few materialized views.

After evaluating options such as CSV file export/import, ETL tools, SAP SLT, Data Services, etc, the favorable option was Oracle Gateway.

To get this Oracle Gateway running was quite a work for the first time. Therefore this blog, to help others.

The way it should works is :-

From within oracle database if a sql statement like SELECT * FROM DUMMY@H1X; is fired, it should bring the data from SAP HANA database (H1X).

First some basics, which is important to understand.

How does it work?

SQL commands are fired from a oracle database (sqlplus), which will reach out for Oracle Gateway via DBLINK > tnsnames.ora

Oracle Gateway will have parameter in its init<sid>.ora file and loads unixODBC libraries from its LD_LIBRARY_PATH.

unixODBC will load HANA odbc drivers and goes through DSN setting to read data from HANA database.

Meaning Oracle DB > DBLINK > tnsnames.ora > Oracle Gateway > unixODBC > Data source DSN > HANA DB

Step by step - How to setup the above scenario

Step 1 - First step is to make unixODBC working

Installed unixODBC rpms (both 32 and 64 bit) on RHEL machine where you will run Oracle Gateway.

unixODBC-2.2.14-11.el6.x86_64

unixODBC-devel-2.2.14-11.el6.i686

unixODBC-devel-2.2.14-11.el6.x86_64

unixODBC-2.2.14-11.el6.i686

Step 2 - Install SAP HANA client (64bit) on RHEL machine where you will run Oracle Gateway. Please refer to the HANA client installation guide.

Step 3 - Create /etc/odbc.ini   contents looks like below

[H1X]

Driver=/usr/sap/hdbclient/libodbcHDB.so

ServerNode=serverhana:30015

Step 4 - Install Oracle Gateway software. You may please google to find step-by-step on how to install using Oracle Universal Installer.

Step 5 - Set environment variable of user running oracle gateway. In my case it is

     LD_LIBRARY_PATH=/usr/lib64:/usr/sap/hdbclient:/oracle/BW1/112_64/lib

Its very important to have 64bit unixODBC libraries path (/usr/lib64/libodbc.so) in LD_LIBRARY_PATH because we run RHEL x64 and Oracle 64 bit software.

Step 6 - Create init<sid>.ora. In my case I will call this as dg4odbc (initdg4odbc.ora). The content should like like below.

HS_DB_NAME = H1X

HS_FDS_CONNECT_INFO = H1X                                   <===== This is the DSN name that comes from step 2 /etc/odbc.ini

HS_FDS_SHAREABLE_NAME = /usr/lib64/libodbc.so

HS_LANGUAGE=AMERICAN_AMERICA.UTF8

HS_NLS_NCHAR=UCS2

HS_FDS_TRANSACTION_MODEL=READ_ONLY

set ODBCINI=/etc/odbc.ini

Step 7 - Create listener.ora

LISTENER =

(ADDRESS_LIST=

(ADDRESS=(PROTOCOL=tcp)(HOST=oragwhost)(PORT=1551))

)

SID_LIST_LISTENER=

  (SID_LIST=

   (SID_DESC=

    (SID_NAME=dg4odbc)

     (ORACLE_HOME=/oracle/BW1/112_64)

      (PROGRAM=dg4odbc)

       (ENV="LD_LIBRARY_PATH=/usr/lib64:/oracle/BW1/112_64/lib:")

      )

  )

Step 8 - Start listener

lsnrctl start

Step 9 - Lets first test unixODBC is working

Login as user which will run oracle gateway and check LD_LIBRARY_PATH (refer step 4 above) and use below commands.

isql -v <DSN name from step 2 /etc/odbc.ini> <hana user name> <password>

For example isql -v H1X SYSTEM password

+---------------------------------------+

| Connected!                            |

|                                       |

| sql-statement                         |

| help [tablename]                      |

| quit                                  |

|                                       |

+---------------------------------------+

SQL> select * from dummy

+------+

| DUMMY|

+------+

| X    |

+------+

SQLRowCount returns 1

1 rows fetched

If you see these output you are half way through. unixODBC is working.

Now its time to work on oracle database from where data will be read with SELECT statements.

Step 10 - Add entries in tnsnames.ora   In my case it will look like below.

dg4odbc  =

  (DESCRIPTION=

   (ADDRESS=(PROTOCOL=tcp)(HOST=oragwhost)(PORT=1551))

   (CONNECT_DATA=(SID=dg4odbc))

   (HS=OK)

  )

And test it with tnsping

Step 11 - Create DB link in oracle database and run a SELECT command.    Commands looks like this.

CREATE PUBLIC DATABASE LINK H1X CONNECT TO

"SYSTEM" IDENTIFIED BY "password" USING 'dg4odbc';

SQL> select * from dummy@H1X;

DUMMY

--------

X

Hope this helps some one in need !

Introduction

With this quick blog I am explaining how to implement a supplier rating application using the Text Analysis capabilities of SAP HANA. I am using Sentiment analysis feature to rate a particular product and suppliers based on the customer feedback on various web sites .

Assumptions

We have the customer feedback collected from suppliers and arranged in the following format. (data is copied from multiple online suppliers. Product names and supplier names are masked )

Modelling your source Table

     Now we need to load the data into SAP HANA. For that, We are creating a table of the same structure .

After once the data is loaded into the source table, we are creating a full text index on the table for the FeedbackText column using the standard configuration "EXTRACTION_CORE_VOICEOFCUSTOMER".

Voice of the customer content includes a set of entity types and rules that address requirements for extracting customer sentiments and requests. You can use this content to retrieve specific information about your customers' needs and perceptions when processing and analyzing text. The configuration involves complex linguistic analysis and pattern matching that includes processing parts of speech, syntactic patterns, negation, and so on, to identify the patterns to be extracted.

Once the index is created , one new columnar table will be created in the same schema where the source table is created. This table will be having the tokens as well as the corresponding sentiments values .

Now generating some visualizations on the generated sentiment data.

SAP Lumira For Reporting

Now we have the sentiment data available in our table. That means, we have the product details, Supplier details, and the type of feedback given by the customers . Now we will generate dashboard(Real time dashboards on LUMIRA if we integrate it with live feedback data on multiple web sites) .

And I love this part

(This blog is created for DataGeek ).

Sreehari

Hadoop offers large-scale, low-cost distributed storage called Hadoop Distributed File System (HDFS). When a large file is stored in Hadoop, it is broken into multiple fixed-size blocks and replicated across data nodes. Since the redundancy is on the small size block level, the recovery is faster and distribution is optimal. In order to process these blocks coherently, Hadoop introduced a programming paradigm called MapReduce (MR). By integrating with Hadoop, you can combine the in-memory processing power of SAP HANA with Hadoop’s ability to store and process huge amounts of data, regardless of structure.

Before SAP HANA SPS09, you can use the Smart Data Access to create virtual tables through hive ODBC driver for data federation scenarios. Start from SPS09, SAP HANA supports the integrations with Hadoop MapReduce (MR) jobs written in Java. You can create new type of User Defined Function with the direct access to HDFS and the vUDFs(Virtual User Defined Function) can be invoked from SQL directly that help customer to reuse their investments in MapReduce and solve the problems don’t fit the typical Hive usage patterns via SDA.

Here is the architecture diagram of the integration: 

1. Install Hadoop adapter controller

You should have your Hadoop system installed already. SAP has created an adapter that can be installed as a delivery unit in HANA XS Engine and will be pushed to Hadoop later. This installation has been done in the system but it is good for you to understand how it works.

The controller can be downloaded from SAP Marketplace. After that, you need to assign the sap.hana.xs.lm.roles::Administrator role to your HANA user then start the HANA Application Lifecycle Manager to import it as a delivery unit.

HANA Application Lifecycle Manager URL is like below, replace the host and instance no. After login, click on Delivery Unit->Import->Browse and locate the tgz file and import it.

http://<yourhost>:80<instanceNumber>/sap/hana/xs/lm/

In the Hadoop side, the controller.jar should have been deployed at /sap/hana/mapred/<Revision>/controller/package in HDFS.

You will also need to put these two jar files at the path /sap/hana/mapred/lib in HDFS:

2. Create the MapReduce java project

You can ask to download the project from this blog. If you want to start from scratch, below are the steps.

In HANA Studio/Eclipse, create a new Java project, give it a name song_count, add the following jar files into a subdirectory lib and add them into class path of the project. You should see something like below after that.

Create the Mapper class with a name SongCountMapper.java in the package com.sfp and then copy the source code below. In the mapper, we find out the songs for each artist.

packagecom.sfp;

importjava.io.IOException;

importorg.apache.hadoop.io.IntWritable;
importorg.apache.hadoop.io.Text;
importorg.apache.hadoop.mapreduce.Mapper;

publicclassSongCountMapperextendsMapper<Object, Text, Text, IntWritable> {

    privatefinalstaticIntWritableone=newIntWritable(1);

    @Override
    publicvoidmap(Objectkey, Textvalue, Contextoutput)throwsIOException,
             InterruptedException {
         String[]song=value.toString().split(",");
        output.write(newText(song[3]),one);
     }

}

Create the Reducer class with a name SongCountReducer.java in the package com.sfp and then copy the source code below. In the reducer, we aggregate the number of songs for each artist.

packagecom.sfp;

importjava.io.IOException;
importorg.apache.hadoop.io.IntWritable;
importorg.apache.hadoop.io.Text;
importorg.apache.hadoop.mapreduce.Reducer;

publicclassSongCountReducerextendsReducer<Text, IntWritable, Text, IntWritable> {

    @Override
    publicvoidreduce(Textkey, Iterable<IntWritable>values, Contextoutput)
            throwsIOException, InterruptedException {
        intcount= 0;
        for(IntWritablevalue:values){
        count+=value.get();
         }
        output.write(key,newIntWritable(count));
     }

}

Create the Application class with a name SongCountApplication.java and then copy the source code below. In the application, it incorporates the mapper and reducer, launch a MapReduce job the process data from the input path /sfp/song/data and put the result into path /sfp/job_result in HDFS.

packagecom.sfp;

importorg.apache.hadoop.conf.Configuration;
importorg.apache.hadoop.conf.Configured;
importorg.apache.hadoop.fs.Path;
importorg.apache.hadoop.io.IntWritable;
importorg.apache.hadoop.io.Text;
importorg.apache.hadoop.mapreduce.Job;
importorg.apache.hadoop.mapreduce.lib.input.FileInputFormat;
importorg.apache.hadoop.mapreduce.lib.input.TextInputFormat;
importorg.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
importorg.apache.hadoop.mapreduce.lib.output.TextOutputFormat;
importorg.apache.hadoop.util.Tool;
importorg.apache.hadoop.util.ToolRunner;


publicclassSongCountApplicationextendsConfiguredimplementsTool{

    publicstaticvoidmain(String[]args)throwsException {
        intres= ToolRunner.run(newConfiguration(),newSongCountApplication(),args);
         System.exit(res);     
     }

    @Override
    publicintrun(String[]args)throwsException {

         Jobjob= Job.getInstance(newConfiguration());
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(IntWritable.class);

        job.setMapperClass(SongCountMapper.class);
        job.setReducerClass(SongCountReducer.class);

        job.setInputFormatClass(TextInputFormat.class);
        job.setOutputFormatClass(TextOutputFormat.class);

         FileInputFormat.setInputPaths(job,newPath("/sfp/song/data"));
         FileOutputFormat.setOutputPath(job,newPath("/sfp/job_result"));

        job.setJarByClass(SongCountApplication.class);

        job.submit();
        return0;
     }

}

Compile the java code and make sure there is no syntax error. The project should look like below now.

3. Upload the MapReduce Jobs Archive into HANA repository

In order to let HANA aware of the MapReduce code, we need to upload the jar file into HANA repository. HANA Studio can help to generate the jar file on the fly from your source code.

We firstly need to create a new package in HANA repository that we can store the jar file. Go to HANA repository(Create a new one if not created), create a new package sfp.song. Create a new general project in HANA Studio/Eclipse named song and share the project with the HANA repository. You should see a project like this after that. The project is really a placeholder for the MapReduce job archive, it is not necessary to create the .xsapp or .xsaccess file like other XS-based project deployed in HANA repository.

Select the project, click New—>Other.. and click SAP HANA—>Database Development—>Hadoop MR Jobs Archive in the pop-up window.

Click next and enter the file name as song_count then click next.

Select the Java project song_count, put the target Schema as your Schema, here is SFP and click Finish.

That will create a song_count.hdbmrjobs file, activate it and you will see it has been uploaded as a jar file in HANA repository by executing this query.

4. Create the Remote Data Source

Now it’s the time to create the remote data source by running the SQL statement below in SQL console of HANA Studio. You will need to replace the <FQDN> with your own Full Qualified Domain Name of your host that you can find it by running command hostname -f.

CREATEREMOTE SOURCE HADOOP_SOURCE

ADAPTER"hadoop"

CONFIGURATION'webhdfs_url=http://<FQDN>:50070;webhcat_url=http://<FQDN>:50111'

WITHCREDENTIALTYPE'PASSWORD'

USING'user=hdfs;password=hdfs';

Now you should see the Remote Source.

5. Create the Virtual Function

To create the virtual function, you can run the following SQL statement in SQL console of HANA Studio. The returns table structure need to be the same data types as the output structure of the Reducer, the Package System is the one you can find at view “SYS”.”VIRTUAL_FUNCTION_PACKAGES”, the configuration need to specify the input path of the MapReduce job where you put your data files in HDFS and the class name of the Mapper and Reducer.

CREATEvirtualFUNCTIONHADOOP_SONG_COUNT()
RETURNSTABLE("song_artist"NVARCHAR(400),"count"INTEGER)
PACKAGESYSTEM."sfp.song::song_count_job"
CONFIGURATION'enable_caching=true;mapred_jobchain=[{"mapred_input":"/sfp/song/data","mapred_mapper":"com.sfp.SongCountMapper","mapred_reducer":"com.sfp.SongCountReducer"}]'

ATHADOOP_SOURCE;

6. Run the Virtual Function

Now you can simply run the virtual function like below.

SELECT*FROMHADOOP_SONG_COUNT();

It will trigger the MapReduce job and execute it in the Hadoop Cluster and populate the result into the output path of your Reducer, in this Reducer, it will be in /sfp/job_results in HDFS.

And you can find the job records in the job history UI at http://<your_hadoop_host>:19888/jobhistory

If you've ever visited London you'll no doubt have seen those courier cyclists weaving in and out of traffic.  You might have seen some of them hurtling through red lights, mounting the pavement (or sidewalk for American readers) and frightening pedestrians.  Cycling in general is being encouraged by the London city mayor.  Being a risk-averse scaredy-cat myself I got to wondering how safe it is to cycle in London?  It certainly doesn't look very safe, so I set about attempting to quantify how safe - or otherwise - it is using HANA's spatial capabilities in SP8.

The UK government now publish road traffic accident data in a very convenient form.  They provide spreadsheets containing 1.4 million accidents spanning 13 years.  I used some of this data previously to allow accidents near your location to be viewed as heatmaps on your mobile phone.  Back then the accident data needed a license and additional work was necessary to get locations, so the data is much easier to use now.

This article will show you how to get the necessary accident data, tidy it up using Linux tools, then perform some spatial analysis using SQL all in an attempt to measure: how safe cycling is in London?

For the impatient, I can reveal that the data shows no increase in average accident severity if you're involved in a cycling accident in London vs the rest of the UK.  Good news for cyclists, then.  Perhaps this is because average speeds in the capital are lower, or perhaps my amateur statistics analysis is flawed.  But I'm getting ahead of myself.  Let's take this step by step:

1) Define the question we want to answer

2) Source the Data and Understand the Data Model

3) Do the Extract, Transformation and Load (ETL) into HANA

4) Analyse Data using HANA SP8 Spatial capabilities

1) Define the question we want to answer

Ideally we'd like to know "how safe is cycling in London?".  To answer that, it would be reasonable to say what is the probability of a cycling journey having an accident.  That would mean we'd need to know how many journeys there are, including those that were incident free, and more about their nature (distance, weather, lighting).  Data about accident-free journeys seems not so easy to get.  Since we're not working to anyone's spec, let's redefine the question that we will answer to suit the data available.  How about:

If you cycle in London and are in an accident, is it likely to be more serious than in the rest of the UK?

The above is much easier to answer with the available data.

2) Source the Data and Understand the Data Model

The source data is available here: http://data.gov.uk/dataset/road-accidents-safety-data.  The data model is very simple, as shown below:

The tables of interest are Accidents and Vehicles.  The Accidents table holds the worst case severity for an accident and it's location.  The Vehicles table holds all vehicles involved and the field Vehicle_Type = 1 identifies cycles.  In fact, that is all that we need.  This is enough to let us filter on cycles only and average the severity over "in London" and "not in London" buckets based on the accident location.

The last thing to consider here is what sort of spatial model we want to use in HANA.  HANA offers 3 spatial reference systems (SRS) in SP8, each identified by a Spatial Reference ID (SRID):

Most of the articles you see use SRID 4326 (aka the "WGS84" system) which is the system used by GPS and it treats the earth as a sphere.  However in HANA SP8, SRID 4326 does not allow tests of points being contained by shapes.  So in this article we're going to use SRID 1000004326.  This effectively takes the earth as a globe from SRID 4326 and stretches it out onto a flat planar surface.  Distances are distorted (look at Antartica in the south of the above diagram) but tests of "does A contain B" are possible.

Suitable HANA tables to hold the data can be found in this GitHub repository, see the RoadF.hdbdd file.

Ok, now we have understood the data model, we're ready to get some data into the system.

3) Do the Extract, Transform and Load (ETL) into HANA

The data is supplied as three tables, one spreadsheet per table.  The data will make a journey through process steps a) to e) as shown below:

a) Use PSCP to Upload Files

First download the files http://data.gov.uk/dataset/road-accidents-safety-data to your local PC.  Next we need to get the files onto the HANA box.  One method for this I'd not seen till recently is using a tool called PSCP.  The HANA system for this demo was a CAL instance and if you use a CAL instance then you may already use PuTTY to connect to the backend Linux host.  When you install PuTTY on a Windows PC you also get a secure FTP client called PSCP.  PSCP can read configurations you've setup in PuTTY and so it is quite convenient to use to FTP files.

Let's use PSCP from a DOS command line to list some files on the Linux host.  In windows run PSCP from the DOS command line like this:

C:\Program Files (x86)\PuTTY>pscp -ls "HANAonBW":/usr/sap/HDB/home/

In the above, the -ls is the Linix command to list files, the "HANAonBW" is a saved PuTTY config to allow us to login and the /usr/sap/HSB/home/ is the directory on the Linux box.  The PuTTY configs are those you you see in the "Saved Session" in PuTTY here:

Now we are familiar with PSCP, it is easy to do the file transfer.  The syntax is like this: pscp <source file from Windows> <PuTTY config>:<target directory on Linux>:

C:\Program Files (x86)\PuTTY>pscp c:\temp\RTA\AccidentsF.csv "HANAonBW":/usr/sap/HDB/home/

b) Use AWK to convert date format

The data provided contains a date in a format that is not compatible with HANA.  We need to change the format of the date from 14/01/2005 to the HANA format 2015-01-14.  To do this, we're going to use a tool that comes with Linux called AWK.  To do this conversion, we use PuTTY to connect to the Linux backend, then run this series of Linux commands:

This runs pretty quickly, around 8 seconds to convert 1.4 million rows.  Taking each line in turn:

// line count of source file for checking
wc -l < AccidentsF.csv
// Change field 10 be HANA style date
awk -F"," '{OFS=","; $10=substr($10,7,4)"-"substr($10,4,2)"-"substr($10,1,2); print $0}' AccidentsF.csv > AccidentsFT.csv
// line count of target file for checking
wc -l < AccidentsFT.csv = 1494276

The AWK command is quite long and deserves a bit more elaboration:

-F","     - to use a field separator of comma

{         - begin actions

OFS=",";  - Output Field Separator, so that the output fields are also separated by a ,

$10=      - field number 10 will equal...

substr($10,7,4)"-"substr($10,4,2)"-"substr($10,1,2); - some strings manipulated to make new date

print $0; - to print out the whole line to the target file.

}         - end actions

AccidentsF.csv  - the input file

>               - is sent to

AccidentsFT.csv - the T transformed file

The result of the above is a new file called "Accidents FT.csv", with the date formatted as 2015-01-14.

c) Upload via CTL files

This is covered already on SCN, and so following the same principles: use PuTTY, move to directory /usr/sap/HDB/home/ and type:

cat > roadaccF.ctl

import data

into table "ROAD"."roadsafety.data::RoadF.AccF"

from '/usr/sap/HDB/home/AccidentsFT.csv'

record delimited by '\n'

field delimited by ','

optionally enclosed by '"'

error log '/usr/sap/HDB/home/AccidentFTErr.txt'

Use [CTRL+D] to end creating the CTL file.  Then upload the data using SQL from insiide HANA studio (the necessary target table definitions are in GitHub here):

IMPORT FROM '/usr/sap/HDB/home/roadaccF.ctl';

d) Data Cleansing

Now need a tiny bit of data cleansing.  The data contains a few records without location data, and these need removed:

delete from "ROAD"."roadsafety.data::RoadF.AccFT" where LAT is null and LON is null;

e) Add Spatial Field & Fill it

To add the spatial field to our Accidents table, we cannot yet add that in the .hdbdd file in HANA Studio, instead we have to manually add the field with SQL.  To do this I followed the article by Rafael Babarand copied the data from the existing table to a new table, populating the Spatial Point in the process.  The SQL to do this in Github.

4) Analyse Data using HANA SP8 Spatial capabilities

I spent much time trying to get HANA models to work, SQLScript calculation views to work, and various errors occurred.  The recommendation seems to be to wait for SP9 for fuller integration with HANA models.  Therefore I used pure SQL to do the analysis of the data.

Before doing any SQL, we need to define what "in London" and "outside London" means.  For this I followed Jon-Paul Boyd's excellent blog and used Google Earth to draw a polygon around the area I was interested in:

That polygon is then exported as a series of coordinates which is used in the SQL statement below.  Finally we're ready for some analysis!  This SQL returns the mean severity and variance of all accidents in London that involved a bicycle:

-- Just bicyles in London

select

AVG (T0."SEVERITY") "Avg Severity",

VAR (T0."SEVERITY") "Avg Severity Variance",

SUM (T0."VEHCOUNT"),

SUM (T0."CASCOUNT")

from

ROAD."roadsafety.data::RoadF.AccFT" T0

left outer join "ROAD"."roadsafety.data::RoadF.VehF" T1

on  T0."ACCREF" = T1."ACCREF"

where T1."VEHTYPE" = 1  --vehtype 1 is bicycle

and NEW ST_Polygon('Polygon((

51.69021545178133 -0.1000795593465265,

51.68262625218747 -0.1640894678953375,

51.62673844314873 -0.5003652550731252,

51.4687978441449 -0.5003020713080952,

51.37537638345922 -0.2604447782463681,

51.29248664506417 -0.1217913673590465,

51.3298782058117 -0.02055237147410183,

51.32142023464126 0.0993682688423303,

51.34618151800474 0.1346959279977478,

51.46491093248794 0.2133695972971839,

51.54192930978628 0.3296565877570212,

51.62542509952219 0.228648947683745,

51.60811732442631 0.0851277551187013,

51.67901853300752 -0.01341248134237749,

51.69021545178133 -0.1000795593465265

))').ST_Contains("LATLON") > 0;  -- use = 0 for outside London

The results are, for cycling accidents inside and outside London:

Results

Location         Severity Mean    Severity Variance

Inside London     2.86076           0.12851

Outside London    2.81853           0.16485

Remember that lower severity is more serious.  Severity is measured as an integer where 1 = fatal, 2 = serious and 3 = minor.  The results suggest it is better to be involved in an accident inside London because the average severity value is higher (less serious).  Perhaps this is because car speeds are slower.

The next question is, are the above results statistically significant?  Could the difference be by chance alone, or does it reveal a pattern in the underlying data?  This is beyond my (very) amateur statistics knowledge, and although there are plenty samples online about "comparing the mean of two populations" they all focus on taking samples from large populations where variance is not known but here we know every data point.  If anyone with statistics knowledge reads this, I'd be interested to know how to go about comparing these means.

As you probably heard, SAP HANA smart data streaming is a new HANA capability introduced with SAP HANA SPS09.  This is an optional capability that can be added to a HANA system to add high-speed real-time event stream processing, with the ability to capture raw or derived streams in the HANA database as well as to perform real-time complex event processing for low latency (sub-second) "sense and respond" scenarios.

This white paper provides a technical overview of smart data streaming,  how it can be used, how it scales, and the basic features of the server and the CCL language.  But here I'd like to provide a quick overview of the six steps you need to take to put this new capability to work.

1. Download it and install it in your HANA system landscape

Smart data streaming is an optional component and is not installed by default on a HANA system. Download the smart data streaming from SAP Service Marketplace (look under "H" for HANA, and then you'll see SAP HANA smart data streaming").  This HANA Academy tutorial will show you how.

Note that smart data streaming requires and additional license.  If you aren't licensing for smart data streaming, talk to your SAP representative.

2. Download and install the streaming plug-in for HANA Studio

From the "support packages and patches" section of Service Marketplace.  Look under "H",  go to SAP HANA smart data streaming, and then "Comprised software component versions" and choose "Smart Data Streaming Studio 1.0"

Watch one of these HANA Academy tutorials for installation instructions:  this one is for installing the streaming plugin when you install HANA Studio;  this one is to add  the streaming plugin to an existing HANA Studio installation.

Important: you need to be using HANA Studio 2 (the new version delivered along with HANA SPS09)

You'll then want to follow the steps in these tutorials  to configure the studio plugin, include connecting it to a streaming server, and then add a data service definition so that your streaming project(s) can query and write to the HANA database

3. Build a streaming project

Streaming projects are written in CCL - continuous computation language - which is the event processing language used by smart data streaming. CCL is derived from SQL - so it's easy to learn, and it also includes a powerful scripting language called CCL Script that can be used to define complex operations that go beyond basic SELECT statements.

The streaming plugin for HANA Studio includes both a CCL editor as well as a visual editor - you can work in either and switch between them at any time to view and edit your project. Check out this HANA Academy tutorial that takes you through the steps of building  a simple project using the streaming visual editor in HANA studio.

4. Connect your project to one or more data sources

Smart data streaming includes a number of input and output adapters that can be used to attach streaming projects to data sources and data destinations.  These include adapters for message buses, web services, files, sockets, email and others.  It also includes a web service provider that exposes streams as web services, with support for REST, SOAP and WebSockets, allowing client applications to easily connect via http.

And the HANA output adapter is used to connect any stream or window in a streaming project to a HANA table, such that the data flows directly from the stream into the connected table in the HANA database - continuously.

5. Test your streaming project

The streaming plugin for HANA Studio includes two perspectives, the streaming development perspective and the streaming run-test perspective. The latter has a number of tools to make it easy for you to test your project before you deploy it on a production system.  This tutorial video takes you through the most basic steps, using the playback tool to stream in test data from a file and the stream viewer to view the output.  Other tools include:  manual input, event tracer, performance monitor (for tuning) and a debugger (set breakpoints)

6. Deploy your streaming project

When you're finished testing your project,  while you can run it from the studio, if you are connected to a streaming server,  more typically for a production system you would put it into the HANA repository and then activate it from there or include the streaming project(s) in a HANA Delivery Unit.

Visit the Smart Data Streaming Developer Center for more information or to ask questions or share your ideas.

I have posted an idea few days back to have the ability to customize Refresh and Save button in EPM Add-in as many of us faced the requirement to customize these 2 buttons based on some condition.

Please go through the below link for more information.

Ability to customize Refresh and Save button in EPM Add-in : View Idea

Help me to improve this idea with your valuable feedback and suggestions.

Thank you..

Hello,

I have posted an idea to refer multiple members of a page axis dimension to other sheets in a workbook. Please help me to improve this idea with your valuable feedback and suggestions.

Please go through the below link for more information -

Refer multiple members of a Page Axis dimension : View Idea

Thank you..

Regards,

Meenu

Hi all,

I think this an simple idea as many of us will be getting confused while working with transactions like  for example while working with smartforms we may get confused on bar code creation and uploading image in smartforms etc.

Suggestions while typing in User command depending upon the starting letter of the TCODE.

for example : What do I mean is in google when we are searching for anything we get autocomplete suggestions in the down. In the same way it will be useful for us if SAP will provide suggestions for transcation codes as it is little bit confusing bewtween some TCODEs.

If this ides sounds good to you then you can click on below link to vote for this:

Provide TCODE Suggestions to users when user types TCODE in user command : View Idea

Thanks.

Sneha Jadhav

In SPS 09 the unit test framework XSUnit (which bases on open-source framework Jasmine) was introduced by SAP. By using this tool server-side JavaScript unit tests can be created for XS applications. As well, the mocking framework Mockstar was introduced which permits to mock database objects or substitute tables/views by stubs. In interaction both tools make it possible to write unit tests for complex database objects like attribute views, analytic views, calculation views and procedures.

This blog guides through the initial difficulties to draft the first unit tests. Furthermore, it examines advanced aspects like measuring the code coverage and debugging unit tests.

Prerequisites

All relevant test tools/frameworks have been bundled by SAP in delivery unit HANA_TEST_TOOLS which is not automatically deployed but available as non-automatic content. For us it worked to download HANATESTTOOLS.tgz and deploy it using the HANA Studio (REPO.IMPORT privilege is required for this activity). According to the SAP development documentation it is possible as well to deploy the delivery unit via Application Lifecycle Management.

Note: After installation the test tools are available in package sap.hana.testtools. In HANA Studio the package content can be inspected in Repository Browser or Project Explorer. On the opposite, the System view will show “hidden objects” since it only displays objects of type Analytic View, Attribute View, Calculation View, Analytic Privileges, Decision Tables, and Procedures.

The following privileges and roles are required to create and run unit tests:

• User must be a standard user (since stub tables are always are created within the user’s schema).
• Select permission must be granted on tables/views under test (in schema where the original tables/views reside).
• (optional) For viewing and updating unit tests the following privileges must be granted on the package where the unit tests reside:
  
  REPO.READ
  REPO.EDIT_NATIVE_OBJECTS
  REPO.ACTIVATE_NATIVE_OBJECTS
  REPO.MAINTAIN_NATIVE_PACKAGES
  
  Additionally, the execute privilege on SYS.REPOSITORY_RESTmust be granted to access the HANA repository.
• (optional) Roles sap.hana.xs.ide.roles::EditorDeveloper and sap.hana.xs.debugger::Debuggerare required for using the web-based development workbench and debug unit tests while runtime.
• (optional) Role sap.hana.testtools.common::TestExecute is required for using the code coverage facility. (Even if this rule is designed for another purpose it is the only role which grants SELECT on the SAP_HANA_TEST schema.)

Writing Unit Tests

Unit tests are supposed to be created as XS JavaScript Library files (suffix .xsjslib) in HANA Studio. The basic syntax is explained in detail on the Jasmine project page. Simplest example:

/*global jasmine, describe, beforeOnce, it, expect*/
describe("test suite", function() {       beforeOnce(function() {              // called before running the test suite       });       it("unit test", function() {              expect(0).toEqual(0);       });
});

describe() implements a test suite and it() implements one unit test within that suite. The comment in the first line is specific for HANA. It must be added to each unit test file and tells the HANA Studio to consider the listed functions to be available. expect() brings along many functions to implement the unit test assertions.

A great variety of unit tests examples is provided with the test tools demos which can be found in package sap.hana.testtools.demos. Thus, this blog leaves it with a pretty simple unit test example. The object under test is an attribute view (AT_EMPLOYEE) which joins tables EMPLOYEE and DEPARTMENT with an outer join:

Note: Both, the attribute view as well as the unit test file must be located in a package which belongs to an XS Application.

When it comes to tests on views it’s mandatory to manipulate the content of referred tables in order to compare the view’s results afterwards with expected values. Certainly, that contradicts the idea of unit tests which are supposed to  test objects isolated. To solve that issue the Mockstar framework can be used to create a temporary copy of the view under test and replace the originally referenced tables with stub tables. These stub tables can be manipulated without worries. The following code would be used to isolate AT_EMPLOYEE:

var testEnv = mockstarEnvironment.defineAndCreate({    targetPackage : 'UnitTesting.temp.' + mockstarEnvironment.userSchema,    schema : 'UNIT_TESTING', // original schema    model : { // object under test        schema : '_SYS_BIC',        name : 'UnitTesting/AT_EMPLOYEE'    },    substituteTables : { // tables to be substituted        "empl" : 'EMPLOYEE',        "dept" : 'DEPARTMENT'    }
});

It’s common practice (within SAP’s demo unit tests) to do that isolation within the beforeOnce() function. After running the unit test for a first time the object under test will be copied into the specified target package. Its source tables will be substituted by identical copies of the original tables which are located in the user schema of the user who ran the test.

Note: It’s not possible to configure the target schema. The substitute tables will always be created in the user’s schema who run the test. If the original tables are located in the same schema (of the testing user) the unit test execution will cause the original tables to be deleted!

Note: Objects under test, in our example the attribute view, are taken from schema “<definition.schema>_SYS_BIC”, since that schema contains all activated objects.

A first, simple unit test could be one which truncates the employee table and checks if the view is running and is returning an empty result set:

// truncate employee table (substitute)
testEnv.clearTestTables([ 'empl' ]);

// check if the view’s result set is empty
sqlExecutor = new SqlExecutor(jasmine.dbConnection);
var actual = sqlExecutor.execQuery('select * from ' + testEnv.getTestModelName());
expect(actual).toMatchData({}, [ "ID" ]);

Running Unit Tests

Because XS applications are run on the HANA server unit tests can’t be started out of HANA Studio. Before a unit test can be run it must be activated and then be run on HANA server using HANA’s web frontend which is accessible in default setup on URIhttp://<hostname>:80<instance>.

On the one hand, on HANA’s web frontend the test runner tool can be used. It is contained in the HANA test tools and does accept parameters which define the target unit test. It searches <package> for unit test with name <pattern>:

On the other hand, unit tests can be run from HANA’s Web-based Development Workbench:

Note: Microsoft Internet Explorer 10+, Mozilla Firefox, and Google Chrome are supported.

Test Data

The XSUnit framework brings along two mechanisms to populate substitute tables with test data. These test tables can be populated with single test records. With regard to the unit test example above the following snippet demonstrates how to insert single records:

testEnv.fillTestTable('empl', {       ID : 123,       NAME : 'John Doe',       DEPARTMENT_ID : 1
});

On the other hand, substitute table can be populated from CSV files which are available in the HANA repository. Thereby, the CSV properties and source package must be defined within the Mockstar environment definition, the table load is done by a separate command which can be placed anywhere in the unit test suite:

testEnv = mockstarEnvironment.defineAndCreate({ // only supplement for definition!     ...     csvPackage : "UnitTesting.object",     csvProperties: {         separator: ";",         headers: true,         decSeparator: ","     },     ...
});

testEnv.fillTestTableFromCsv("empl", "employee.csv");

Expected values can be matched against the actual view output as illustrated by the following example:

var expected = {     ID : [ 123 ],     NAME : [ 'John Doe' ] ,     DEPARTMENT_ID : 1
};
var actual = sqlExecutor.execQuery('select * from ' + testEnv.getTestModelName());
expect(actual).toMatchData(expected, [ "ID" ]);

Truncating substitute tables works for single as well as for all dependent tables:

testEnv.clearTestTables([ 'dept' ]);
testEnv.clearAllTestTables();

Code Coverage

Since SRS 09 the open-source code coverage library BlanketJS is integrated into the HANA Test Tools. It measures to which extent JavaScript code is covered by unit tests. Unfortunately, for unit tests on model views (attribute views, analytic views, calculation views and procedures) this approach can’t be applied.

Debug Unit Tests

When running unit tests for a first time you might miss privileges or have semantic failures in your code. If that occurs you will be provided with stack traces and error messages. If those information are not sufficient to understand the issue there are options to debug server-side JavaScript code. Thereby both, your unit test code as well as the HANA test tool libraries can be debugged (to a certain extent).

For enabling the debugging functionality followthe instructions on help.sap.com. These instructions will guide you (1) to deploy delivery unit HANA_XS_BASE and (2) add section “debugger” to “xsengine.ini”. Afterwards the debugging can be initiated from HANA Studio or from Developer Workbench.

Regarding the example unit test above a possible scenario for receiving a meaningless error message is when you miss privileges to run the analytic view. In that case error “258 - insufficient privilege” will be returned. When running the unit test from Developer Workbench the following output will be shown:

To initiate debugging (1) make sure that the target session for debugging is THIS_SESSION in editor settings, (2) set a breakpoint on the code line were you want to start the debugging by clicking on the line number and (3) click on link “Create test report” which is located bellow the test results. The link will re-run the unit test in a new browser tab. Because you set a breakpoint the execution will stop at the specified code line and the debug control will appear in the Workbench:

Use the Step In (F3) and Step Over (F4) buttons to navigate through the code.

Note: When it comes to debugging the Development Workbench behaves different depending whether the file is a standard JavaScript file or an Unit Test file. Standard JavaScript files can be run immediately in debug mode by setting breakpoints and run the script/library. Unit Test files must be run in another browser tab (via TestRunner.xsjs) to debug that file in Workbench.

Scenario

I chose to model fire propagation across the suburban landscape. Obviously, since I’m not a subject matter expert in that, I didn’t try to make the model actually useful for real world predictions. Rather my aim was to pick a realistically looking aspect of fire propagation and use it to showcase spatial analysis capabilities of HANA.

So, leaving aside all other factors of fire propagation, let’s assume that in an event of fire the wind may carry away burning particles, which, when falling to the ground, may start new fires.

Spatial data used

I used three sources of data, all available for free from Australian Bureau of Statistics:

• - boundaries of postcodes in New South Wales,
• - boundaries of Australian Statistical Geography Standard (ASGS) mesh blocks,
• - census data associated with ASGS mesh blocks: population, number dwellings, type of land use.

I imported all that data into HANA using IMPORT statement and using SRS 1000004326, which would allow to measure distances – more about that in the next section.

Flat Earth, Round Earth

The Earth has an ellipsoid shape, hence one geographical degree means different distance on the equator and in Sydney. That makes using degrees difficult for measuring distances. HANA provides a few pre-set Spatial Reference Systems (SRS), falling into either “flat Earth” or “round Earth” category. Measuring distances only works in “flat Earth” one.

In a fully-fledged GIS system, there would be some sort of transformation available to convert a shape from linear to polar units and back. HANA lacks that functionality: the ST_TRANSFORM() function will serve some other purpose (and only in SPS10), and the ST_TRANSLATE function has just been added to the feature list. One is left with either the option to implement Vincenty’s formulae, or assume that 1 polar degree has the same value within a local map. For example,

SELECT new st_point(151,-31,1000004326).st_distance(new st_point(151.001,-31,1000004326), 'meter')

  as v_dx

fromdummy

should give a metric value of the longitude polar unit in the near proximity of the point -31 lat 151 long.

My understanding of the example above is that behind the scene, HANA converts geographic degrees into some internal “flat Earth” coordinates (of SRS 1000004326) and calculates the distance. The values of those internal coordinates are not made available to the user.

Using one of these options, one can use HANA math functions and build a shape in linear system, translate it to geographic degrees and save to the database, or just pass back to SAPUI5 application as GeoJSON().

Interface

I picked Google Maps API, as it seemed to have the easiest learning curve for me. Other choices would be probably Nokia Here maps and SAP Visual Business. All these APIs can be integrated into SAPUI5, and SAP Visual Business, in fact, has its own controls in SAPUI5 framework.

To make the application portable between map providers, I moved all calculation and measuring logic to database procedures and used the Google Maps API only for geospatial data visualisation and for user input capture.

Google Maps visualises javascript objects of class google.maps.Data.Feature, where a Feature has the ID, some properties and, lastly, its geometry represented as GeoJSON. Conversion of HANA internally stored spatial objects into GeoJSON has to happen in database procedures, and Features can be built either on the server (XSJS) or on the client (SAPUI5) side.

For my app, I implemented visualization of postcode containing a point and attached it to a Click event on the map:

Simple spatial functions

I built a simple tool to measure distances on the map. The google.maps API captures two consecutive Click events, then HANA uses ST_DISTANCE() spatial function to measure the distance between those two points.

Similarly, using distance measurement functions within an arctangent expression, I calculated the simulated wind direction from two consecutive Click events.

Modelling smoke dispersal

For a particle to be carried X units away from the fire along the wind direction, there is some probability associated with that event. That probability has normal distribution along X. Then, there is some probability for a particle to diffuse Y units across the wind direction -- that probability also has normal distribution, but obviously with different parameters; in fact, there are equations describing that along/across dispersion.

For the purpose of further work, I changed those equations to outline a patch on the surface, that encompasses all probabilities below some threshold (say, 1-2 sigmas) and created a HANA stored procedure to calculate an ellipsoid-like shape of that patch.

In this scenario, an area inside the red patch is under significant risk of fire (blue arrow indicates the wind direction).

Spatial join

I wanted to find out a) what ASGS mesh blocks would fall into the risk area and get some statistics about them, b) to what postcodes those mesh blocks belong. Both tasks would require joining tables not by usual alpha/numeric  fields, but using a spatial relationship between shapes in records. Examples would be “A within X meters from B””, “A overlaps/touches B”, “A is covered by B” etc.

A simplified way to use spatial joins would be in a Calculation View, and there is a SAP tutorial for that. An SQLScript example for “intersection join” would look like this:

SELECT t1.mb_code11 as mesh_code,

t2.POA_2006 as postcode

from"SPATIAL"."T_SHP_MESH"as t1

innerjoin"SPATIAL"."T_SHP_POSTAL"as t2

    on t1.shape.st_intersects(t2.shape) = 1

Here, I added a SAPUI5 Table that would interactively select the row with the data about a mesh block the user clicks on:

Performance considerations

HANA is undeniably fast, but my experience with spatial analysis in HANA so far indicates that there is some amount of optimisation to be done. I may be pushing HANA too far with the amount of data I loaded, but since it’s just one Australian state I doubt that.

So, performance degrades dramatically with increased complexity of spatial features being analysed, increased both in terms of the number features and number of vertices in features. One should be careful, for example, with using ST_BUFFER(), as it produces a rather finely shaped circle polygon with the number of vertices that can totally choke the database. It would be good of SAP to provide functionality to reduce the number of vertices in a shape, I remember having that in ESRI’s Arc/INFO.

Another idea that proved useful was to build a “spatial index” of loaded shapes, for example by creating a rectangle containing each shape:

insertinto"SPATIAL"."T_SHP_MESH_INDEX"

( select mb_code11, mb_cat11,

         new st_polygon('Polygon ((' ||

               shape.st_xmin() || ' ' || shape.st_ymin() || ', ' ||

               shape.st_xmin() || ' ' || shape.st_ymax() || ', ' ||

               shape.st_xmax() || ' ' || shape.st_ymax() || ', ' ||

               shape.st_xmax() || ' ' || shape.st_ymin() || ', ' ||

               shape.st_xmin() || ' ' || shape.st_ymin() || '))')

               as shape

               from"SPATIAL"."T_SHP_MESH")

Rough and slightly redundant selection of features might be made using that “spatial index” and then fine selection with real shapes would be performed on a subset. In my case, this trick reduced selection time from eternity to a few seconds.

Possible extension

This model has some potential for further extension for predictive analysis.

The ASGS mesh blocks already provide land use type and population density, which may give a clue on how fire-prone a mesh block is: say, a high-rise block is less prone to catch a fire than a bushland piece. Further, some historical data on detected fires, with their coordinates, time of detection, wind parameters etc. could be used to derive spatial relationships (clustering? distances?) between historical fires and build a more thorough predictive model.

CONTENTS:

SERIES DATA TABLE CREATION and GENERATE TIMESTAMP DATA

SERIES DATA TABLE vs REGULAR TABLE – STORAGE and COMPRESSION

EXPLORING SERIES DATA BUILT-IN FUNCTIONS

PREREQUISITES:     

• Series Data Column Table "STOCKS_DATA_SERIES" has to be present in Schema “SERIES_DATA".

[CSV Files (STOCKS_DATA_SERIES.csv) is attached in this Post,

Using Flat File Import Create tables "STOCKS_DATA_SERIES"  in schema “SERIES_DATA" in your landscape.]

Exercise 1:  Create & Compare the Series table with Column table

Verify both normal column table and series Table.

Normal Column table --> TS Column memory size --> 5 MB

Series Table --> TS Column memory size --> 10 KB

Now You can understand How efficiently Series Table stores the data for  Time values.

Exercise 2:  Series Table Built-in and Analytic Functions:

Summary:

You have completed the exercise!

You are now able to:

1)  Create the Time Series Table.

2)  Understand the Storage of Series Data Table.

3)  Usage of Series Data Built-in Functions.

Hello,

Find below SAP HANA SQL Script to get Current Quarter Beginning Date, Last Quarter Beginning Date, and Next Quarter Beginning Date.

Suppose you have business requirements based on some date column you need to do some manipulation or calculation based on Current Quarter Beginning Date, Last Quarter Beginning Date, or Next Quarter Beginning Date. In below example you can Change CURRENT_DATE From actual table date field and DUMMY with your actual table according to your requirements, for example i took dummy and current_date.

SELECT * FROM (SELECT CASE WHEN (SUBSTRING (QUARTER (TO_DATE(CURRENT_DATE, 'YYYY-MM-DD'), 1),6,2) = 'Q1')

  THEN TO_DATE(YEAR(CURRENT_DATE))

            WHEN (SUBSTRING (QUARTER (TO_DATE(CURRENT_DATE, 'YYYY-MM-DD'), 1),6,2) = 'Q2')

            THEN ADD_MONTHS(TO_DATE(YEAR(CURRENT_DATE), 'YYYY'),3)

            WHEN (SUBSTRING (QUARTER (TO_DATE(CURRENT_DATE, 'YYYY-MM-DD'), 1),6,2) = 'Q3')

            THEN ADD_MONTHS(TO_DATE(YEAR(CURRENT_DATE), 'YYYY'),6)

            WHEN (SUBSTRING (QUARTER (TO_DATE(CURRENT_DATE, 'YYYY-MM-DD'), 1),6,2) = 'Q4')

            THEN ADD_MONTHS(TO_DATE(YEAR(CURRENT_DATE), 'YYYY'),9)

            ELSE CURRENT_DATE

            END AS Quarter_Begining_Date

  FROM DUMMY),

  (SELECT CASE WHEN (SUBSTRING (QUARTER (TO_DATE(CURRENT_DATE, 'YYYY-MM-DD'), 1),6,2) = 'Q1')

  THEN ADD_MONTHS(TO_DATE(YEAR(CURRENT_DATE)),-3)

            WHEN (SUBSTRING (QUARTER (TO_DATE(CURRENT_DATE, 'YYYY-MM-DD'), 1),6,2) = 'Q2')

            THEN ADD_MONTHS(TO_DATE(YEAR(CURRENT_DATE), 'YYYY'),-6)

            WHEN (SUBSTRING (QUARTER (TO_DATE(CURRENT_DATE, 'YYYY-MM-DD'), 1),6,2) = 'Q3')

            THEN ADD_MONTHS(TO_DATE(YEAR(CURRENT_DATE), 'YYYY'),-9)

            WHEN (SUBSTRING (QUARTER (TO_DATE(CURRENT_DATE, 'YYYY-MM-DD'), 1),6,2) = 'Q4')

            THEN ADD_MONTHS(TO_DATE(YEAR(CURRENT_DATE), 'YYYY'),-12)

            ELSE CURRENT_DATE

            END AS Last_Quarter_Begining_Date

  FROM DUMMY),

  (SELECT CASE WHEN (SUBSTRING (QUARTER (TO_DATE(CURRENT_DATE, 'YYYY-MM-DD'), 1),6,2) = 'Q1')

  THEN ADD_MONTHS(TO_DATE(YEAR(CURRENT_DATE), 'YYYY'),3)

            WHEN (SUBSTRING (QUARTER (TO_DATE(CURRENT_DATE, 'YYYY-MM-DD'), 1),6,2) = 'Q2')

            THEN ADD_MONTHS(TO_DATE(YEAR(CURRENT_DATE), 'YYYY'),6)

            WHEN (SUBSTRING (QUARTER (TO_DATE(CURRENT_DATE, 'YYYY-MM-DD'), 1),6,2) = 'Q3')

            THEN ADD_MONTHS(TO_DATE(YEAR(CURRENT_DATE), 'YYYY'),9)

            WHEN (SUBSTRING (QUARTER (TO_DATE(CURRENT_DATE, 'YYYY-MM-DD'), 1),6,2) = 'Q4')

            THEN ADD_MONTHS(TO_DATE(YEAR(CURRENT_DATE), 'YYYY'),12)

            ELSE CURRENT_DATE

            END AS Next_Quarter_Begining_Date

  FROM DUMMY);

Thanks

Intro

OAuth2 is finally here  - being delivered with HANA SPS 09. Great! But how to use it and what is it all about with this OAuth hype?

Welcome to a series of posts showing you how to use the OAuth2 authentication mechanism in HANA XS. The mechanism enables you to use APIs from providers like Google, Facebook, Twitter or any one of the others offering this authentication. Of course you will also learn how to authenticate against the HANA Cloud Platform and an ABAP server.

In this first post we will look at the prerequisites regarding your HANA XS knowledge and the theory behind OAuth itself. The next post will show you how to use the delivered Google OAuth configuration package as a first example. Follow up posts become more advanced i.e. describing how to create own configurations, how to debug outgoing requests and the like.

Currently the series contains following posts:

1. OAuth2 authentication using HANA XS – Basics (1) (start off with this one, if you are new to OAuth)
2. OAuth2 authentication using HANA XS – XS OAuth client lib calling Google's API (2) (end to end description in a step by step manner)

I will try to keep things as simple as possible, while still not loosing advanced users in lengthy explanations of details.

Prerequisites

This is a rather advanced topic in the area of SAP HANA XS. So if you are totally new to it (you have never heard of .xsjs, .xsapp, .xsaccess, …), here are a couple of resources to get you started:

• openSAP
• Developer center
• HANA academy

If you already know about these, but aren’t so sure about .xshttpdest or trust stores, you may be better off with this series in the first place:

• Outbound httpS with HANA XS (part 1) - set up your HANA box to use SSL/TLS
• Outbound httpS with HANA XS (part 2) - set up the trust relation
• Outbound httpS with HANA XS (part 3) - call the https outbound service in XS server side JavaScript (xsjs)

After reading the OAuth series you will also understand the XS artifacts .xsoauthclientconfig and .xsoauthclientflavor and where they belong in the big puzzle.

So let's get started with a basic introduction to OAuth.

If you already know about OAuth you may check out the chapter 'The theory behind OAuth 2.0 in HANA XS'  only or skip this post and advance to the next postor any other post in this series:

• OAuth2 authentication using HANA XS – XS OAuth client lib calling Google's API (2)

Why OAuth?

OAuth is an open standard to authorization, but why is it so popular today?

Currently there is a strong trend away from a rich client or heavy weight architecture towards distributed systems with lightweight architectures based on plain text (i.e. JSON) web services. Representational state transfer (REST) is the most popular flavor of this architectural style. And OAuth is a perfect match to this architecture. Why?

Usually a REST web service offers it’s API via Hypertext Transfer Protocol (HTTP), using it’s very standard methods (GET, PUT, POST, …) for actions to perform. This protocol, and hereby REST, is supported out of the box in practically all common coding languages. This ubiquitous support, an easier consumption compared to SOAP / WSDL based web services, the growing number of cloud scenarios and REST being a performing and maintainable architecture (please see REST's implementation constraints for details) furthermore foster this trend.

REST services often are powerful and need an advanced security concept. OAuth is a natural fit, as it

• is specifically designed to work with HTTP
• implements role based authorizations
• supports frequent and easy updates to user accounts
• is a centralized service

What is OAuth?

OAuth2 specifies a protocol enabling client/web applications to act on behalf of a user after the user explicitly granted permissions. Another web application/web service (the resource server) finally carries out the actions a client is allowed to perform. The user approves the actions by telling an authorization server that he trusts the client to do what it is asking for.

The most common way for a client to present itself to a resource server is using a bearer token, which is obtained from the authorization server and stored by the client. OAuth does not specify the token itself – only the protocol how to obtain it. To finally access a resource server the client sends a special http header in the form:

     Authorization: Bearer <TOKEN_VALUE>

The token can be decoded by the resource server only (opaque to the client as well) in order to check whether client and user have permissions to access the requested resource.

This is where scopes (a key aspect of OAuth 2.0) come into play. A scope is an access range. As opposed to unlimited access - as it would be without scopes (e.g. using basic authentication) - OAuth enables an application to call services with exactly the right portion of privileges. The scopes are derived by the resource server solely by decoding the token. Other information can be encoded in the token as well. The most important values (besides the scopes) are client ID, target resource ID and the current user ID.

Prominent examples of OAuth 2.0 authorization servers are Google, Facebook and Twitter. All of them also provide resource servers (respectively Google APIs, graph API, streaming API). You will find plenty more providers at e.g. Wikipedia. Wikipedia also serves a good explanation of OAuth

A real life example

Let's look at a real life example.

The user accesses an application in the browser. This application runs on HANA XS and needs information from another application server, e.g. SAP JAM. To access the data, JAM allows the application to authenticate its users by OAuth. For this purpose, JAM acts as authorization server and issues access tokens for the user. These are consumed by the resource server of JAM to provide access to the required services.

As a prerequisite to access resources, the first check determines whether an OAuth token is already existing for this user. This is not done directly by the application, but by the OAuth 2.0 client functionality of HANA XS. Assuming no token is available, the OAuth client initiates the token flow to the OAuth authorization server. The access token request contains the requested scopes and information authenticating both - the calling system and the end user. The resulting access token response contains the access token and additional information regarding token lifetime and granted scopes. This data is persisted for future re-use in the token storage. Now we have a token. Hence, in a subsequent call the access token is retrieved from the token storage and added to the http request enabling direct access to the resource.

The theory behind OAuth 2.0 in HANA XS

As we now know about OAuth and HANA XS, let’s see how they come together. Before continuing with a concrete example in the next post, we'll look at the theory (which you may skip, if you want to).

To connect to a remote service, an OAuth HANA XS client application uses the destination service. In analogy to the destination service in NW AS JAVA or SM59 in NW AS ABAP, it contains all configuration information required for the remote connectivity, like the target URL.

The application uses the HTTP client which facilitates the HTTP protocol. This client is initialized with an HTTP destination. If the latter is configured to use OAuth, the OAuth client is initialized for the target system. To send the HTTP request to the resource server (e.g. JAM, Google), control is passed over to the OAuth client.

The OAuth client first checks whether an adequate access token already exists for the current user and OAuth client ID combination and that this token is covering the scopes desired by the application. If no token is found, the access token request is prepared and send using a new HTTP client instance (to separate the cookie containers from the end-user's client). It contains the client and user credentials, plus the requested scopes. The list of scopes that is requested is the collection of all scopes as declared in the application configuration for this OAuth client ID. The access token response is then evaluated by extracting the access token and persisting the latter one to the token storage, together with the validity and the list of granted scopes. When finally calling the resource server, the HTTP request is enriched with the valid access token.

Obtaining client and current user's credentials or what about flows?

In HANA XS the client credentials are maintained during the configuration and persisted in the secure store. Depending on the OAuth flow, different ways exist how to obtain the user's credentials.

The most important flow is the authorization code flow. It’s being used by most ID providers and we are going to look at an example in the next post. To receive an access token using this flow the following steps have to be carried out:

1. create a client application at the authorization provider and remember the presented credentials
2. obtain an authorization code from the authorization server using these credentials and by granting permissions in the user consent
3. exchange this authorization code for an access token and a refresh token
4. use the access token to make API calls as long as it is valid
5. get a new access token via the refresh token when the lifetime of the access token is over

Another important flow, being used by e.g. JAM, is the SAML flow. This flow involves the creation of a SAML assertion for a user. The OAuth client initializes the SAML issuer for JAM, which includes the JAM-specific SAML configuration. The SAML issuer than creates a SAML assertion for the current user. This assertion is added to the access token request.

Other OAuth flows, such as Implicit flow, Resource Owner Password Credentials or Client Credentials are currently not supported.

Conclusion

I hope you got properly equipped for the upcoming practical part and start implementing straight away.

Next post

Intro

OAuth2 is finally here - being delivered with HANA SPS 09. Great! But how to use it and what is it all about this OAuth hype?

Welcome to the second post in a series showing you how to use the OAuth2 authentication mechanism in HANA XS. The mechanism enables you to use APIs from providers like Google, Facebook, Twitter or any one of the others offering this authentication. Of course you will also learn how to authenticate against the HANA Cloud Platform and an ABAP server.

In this particular post we are going to implement step by step an end to end XS application calling a Google API (plus).

Currently the series contains following posts:

1. OAuth2 authentication using HANA XS – Basics (1) (start off with this one, if you are new to OAuth)
2. OAuth2 authentication using HANA XS – XS OAuth client lib calling Google's API (2) (end to end description in a step by step manner)

I will try to keep things as simple as possible, while not losing experts in lengthy detail discussions.

Overview

In order to get the whole scenario running I am using a productive HANA cloud instance with revision 91. The instance can be reached from Google servers. This is a necessary prerequisite for Google’s web server application scenario. It is also possible to use OAuth2 with an on premise HANA box, which cannot be reached from the Internet.

We are going to perform the following steps:

• Create a Google project and an OAuth application via Google’s developer console
• Configure a trust store
• Create required XS artifacts (xshttpdest, xsoauthclientconfig, xsoauthappconfig, …)
• Create a xsjs calling a Google API

So let’s get it done!

Authorization server configuration (Google)

In order to make Google API calls later on, we first of all have to register our client application at Google’s developer console. During the registration we receive credentials, we later on use to register instances of our application – every user has to grant access to his data, nevertheless all application instances share the same basic client ID and secret.

This is achieved by:

• Go to the Google developer console
• Log on (you need to create a Google account if you want to user their API)
• Create a project

• Navigate into the project, section credentials (below APIs & auth)
• Click the ‘Create new Client ID’ button

• Leave the default ‘Web application’ and click continue
• Provide your email address and a description
• In the following popup you need to configure the connection from Google to HANA XS
  • Authorized JavaScript origins: Provide the publically available fully qualified hostname of your HANA instance
    • e.g.: https://superoauth.hana.ondemand.com
    • Authorized redirect URIS:
      • Google calls this URI to prevent attacks and to hand over control back to the calling application
      • You can implement your own handler or use SAPs delivered generic handler available at
        • <YOUR HOST>/sap/hana/xs/oAuth/lib/runtime/tokenRequest.xsjs

• E.g.: https://superoauth.hana.ondemand.com/sap/hana/xs/oAuth/lib/runtime/tokenRequest.xsjs

Your Google client configuration should now look like this:

Later on we need the client ID and secret. The other information is already present in your XS box for Google. The developer console provides a convenient ‘Download JSON’ button to save this information to your local machine.

Enable the Google+ API

Another thing we can do right now is the enablement of the API we want to call later on. In this post we will use the Google+ API and this is the way to enable it:

• Navigate to the API link below ‘APIs & auth’
• Search for ‘+’
• Enable the API via the button in the status column

HANA XS configuration

If you want to configure OAuth authentication in XS, several configuration aspects come into play. There are well-known ones as XS HTTP destinations and XS Trust Stores, but also some new ones, namely XS OAuth configuration packages. We are going to create everything from scratch in a package called googleAPI. The only thing being reused is the configuration for calling the Google API (more on this later).

To perform all the tasks your user needs to have several roles:

• RuntimeConfAdministrator
• HTTPDestAdministrator
• oAuthAdmin
• and some standard developer roles
  • sap.hana.ide.roles::EditorDeveloper
  • sap.hana.xs.debugger::Debugger
  • sap.hana.ide.roles::CatalogDeveloper
  • sap.hana.ide.roles::SecurityAdmin
  • sap.hana.ide.roles::TraceViewerNote

Setting up the Trust Store

Google only allows secure calls to their APIs (via HTTPS). So we have to set up a XS HTTP destination using a Trust Store. This Trust Store holds the server certificate – signed by a certification authority. We have to retrieve this certificate and import it into a trust store.

• If your HANA box needs to be security enabled check Outbound httpS with HANA XS (part 1) - set up your HANA box to use SSL/TLSfor instructions.

To retrieve the certificate

(I am using Chrome under Windows 7 in this example)

• browse to: https://www.googleapis.com/
• click the green lock next to the URL
• switch to the tab ‘Connection’
• open the ‘certificate information’

• switch to tab ‘certification path’
• select ‘Geo Trust Global CA’
• click ‘view certificate’
• switch to ‘details’
• click ‘copy to file’

• keep the first option ‘DER encoded binary X.509 (.CER)

• save the file to your local machine

To import the certificate

• Switch over to the XS Admin at: <your host>/sap/hana/xs/admin/
• Change to the Trust Manager

• Add a new trust store via the 'Add' link below the list of trust stores
• Give it a name (e.g. googleAPI)
• Switch to the tab ‘certificate list’
• Click 'Import Certificate'

• Select the exported certificate and import it
• If it looks like this, you are good to continue

Basic application setup

We want to have a folder containing everything we configure and are going to do so via the web IDE:

• open the web IDE <your host>/sap/hana/xs/ide/editor/
• right click on ‘content’
• select ‘Create Application’ with the following parameters
  • Template: ‘Empty application’
  • Package: ‘googleAPI’ (or something else you prefer and are willing to remember )

Disable security checks for this application

For reasons of demonstration we make things as simple as possible and switch off authentication:

• Open the .xsaccess file in the new application and change the authentication

//  FROM
// "authentication": [{
 // "method": "Form"  //  }],
// TO
"authentication": null,

so it looks like

XS HTTP destination

We already created the trust store and imported Google's certificate. Now we link this trust store to the HTTP destination so we retrieve a HTTPS destination.

Using the Web IDE, we do this by creating a new file in our package called

    googleAPIdestination.xshttpdest

with following content:

host = "www.googleapis.com";
port = 443;
pathPrefix = "";
proxyType = none;
authType = none;
useSSL = true;
timeout = 0;

To connect the http destination to the trust store, we go to the XS Admin again.

• Open the XS Admin: <your host>/sap/hana/xs/admin/
• Navigate to your package (googleAPI) via the right arrow next to it
• Select the .xshttpdest file and click the ‘edit’ button

• In the tab ‘authentication details’
  • Select ‘SSL enabled’
  • TrustStore: your configured trust store (googleAPI)
  • ‘SSL authentication type’: ‘Anonymous’

• Save the destination for the moment – we’ll come back to it later on

OAuth Configuration Package

Background

OAuth is an open standard. Due to the fact that it has been in a draft version for quite some time, different providers implemented slightly different solutions. They all are standard compliant, but use e.g. different names for parameters and the like.

HANA XS provides a mechanism to meet the requirements of this fact by providing a basic framework and provider specific configurations. These configurations consist of:

• OAuth client configuration (.xsoauthclientconfig)

here you find: client ID, authentication type, endpoints, …

• OAuth client flavor configuration (.xsoauthclientflavor)

here you find: protocol steps, parameter names and http request location (header, body, …)

SAP delivers templates for some providers (currently Google, HANA Cloud platform, SAP Jam, SAP ABAP). Of course it is possible to create further OAuth client configurations and I strongly want to encourage you to do so.

To link the metadata (config & flavor) to an actual XS application there is a final piece in the puzzle - completing an XS OAuth configuration package:

• OAuth application configuration (.xsoauthappconfig)

          Links metadata to runtime data

And this is how we do it...

Create a XS OAuth client configuration package

Use the Web IDE and create a file with the suffix .xsoauthappconfig in our application package. Details will be configured via the XS admin.

Let’s name the file googleOAconfig.xsoauthappconfig and give it the minimum content required:

{
"clientConfig"    : "sap.hana.xs.oAuth.lib.providerconfig.providermodel:googleV1"
}

Now our XS application is linked to the SAP delivered Google provider model.

• If you are curious, you might want to check the package sap.hana.xs.oAuth.lib.providerconfig.providermodel
  • It contains SAP predelivered OAuth configurations (currently HCP, Jam, ABAP, Google) and gives you an idea of how to develop own OAuth configuration packages

To provide the missing content of the OAuth configuration we switch over to the HANA XS Admin again:

• Open the Admin UI at <your host>/sap/hana/xs/admin/
• Navigate into our package
• Select the HTTP destination configuration (googleAPIdestination.xshttpdest)
• Change to the ‘OAuth details’ tab
• Click ‘Edit’ in the lower right corner
• Click ‘Browse OAuth App Configs’

• In the popup: select our fresh OAuth configuration (googleOAconfig.xsoauthappconfig)
• Save your work

Now this particular HTTP destination will use the OAuth authentication. We still are using the pre delivered default Google configuration, which is not aware of our client ID and secret (which we set up at Google's developer console). Even though it is possible, it is not a good idea to provide this information in the default configuration, as an upgrade of the HANA revision will overwrite our details. This is where the extension mechanism comes in handy:

• In the ‘OAuth Details’ tab of the HTTP destination click: ‘Navigate to OAuth App Config’

As we want to call the Google Plus API later on,  we need to tell the app configuration which scopes (authorizations) we require. To add the scope

• Hit the ‘Edit’ button
• Click the + icon above ‘mandatory scopes’
  • In the popup add ‘https://www.googleapis.com/auth/plus.me’
    • I’ll explain later on, where the scope value is coming from

• Save the XS OAuth application configuration
• And click ‘Navigate to Client Config’

• You will see the default client configuration, which we are going to extend now: click ‘Extend’

• Choose our package and give the extension a meaningful name like: googleOAClientExtension

• Now it is time to provide our Google client credentials and a meaningful redirect URL
  • Click edit and provide the Client ID you downloaded as JSON from Google’s developer console
  • Set the redirect URL to <your host>/sap/hana/xs/oAuth/lib/runtime/tokenRequest.xsjs
  • Save
  • Now click the ‘Store client secret’ link in the lower right corner
    • Provide the client secret from Google’s developer console

That’s it for the configuration part – a no brainer, wasn’t it ?

Let’s go ahead and use the configured HTTP destination in a xsjs.

HANA XS OAuth client

Oauth client library

HANA XS SPS9 comes with an OAuth client library. It is a hidden a little bid… but maybe you want to have a look into package:  sap.hana.xs.oAuth.lib and particularly at file oAuthClient.xsjslib

You will find useful methods for the OAuth scenario over there. I’d suggest you briefly look at the following table and use it as a reference later on so we can use the remainder of this post to finish our end to end scenario by an implementation.

// Example on how to load and initialize the library:  var oAuthClientLib = $.import("sap.hana.xs.oAuth.lib", "oAuthClient");
// The constructor requires package name and file name of configured HTTP destination
 var oaC = new oAuthClientLib.OAuthClient (<folder>, <destfilename>);

Following methods are available:

Example xsjs calling the Google API

Before we go into the implementation, remember we chose to use the Google Plus API by

• Enabling the API in the Google developer console
• Adding the required scope to the XS OAuth app configuration
  • if you haven't done so
    • Open the XS Admin and navigate to our xsoauthconfig
      • Hit the ‘Edit’ button
      • Click the + icon above ‘mandatory scopes’
        • In the popup add ‘https://www.googleapis.com/auth/plus.me’
        • Save you work

Where the scope comes from

Scopes define fine granular access permissions. Depending on the API you want to use, you need to use a particular scope. There are several ways to discover the perfect scopes for the task. I like using Google’s OAuth playground (https://developers.google.com/oauthplayground/ . This is also a very good resource to get a feeling for this API and the OAuth authorization flow itself. Other good resources are Google’s API explorer ( https://developers.google.com/apis-explorer/#p/ ) and of course the according documentation https://developers.google.com/+/api/#auth.

Show me the code!

Okay this was a lot of setup work. Stay tuned just a little longer before we finally implement and execute the application.

For this API, you need to get your google ID. To do so:

• Go to https://plus.google.com/
• Navigate to your profile

• Look at the resulting URL and extract the long number
  • In my case this is: https://plus.google.com/101287807282311627286/posts
  • So my ID is 101287807282311627286
  • Paste this ID into the code for the variable ‘yourGoogleID’

And finally: let's code!

• Please check the comments in the code if you are asking yourself what is going on
• Using the Web IDE, create a file called ‘callGooglePlusAPI.xsjs’
• Paste the following code and execute it via the green arrow or F8

Please enjoy the code – you really deserve it now:

var callGoogle = function() {    // you need to add yourGoogleID right here (get it via https://plus.google.com/ -> Profile -> copy the long number)    var yourGoogleID = "101287807282311627286";    var response;    // the HTTP client to send requests    var oHttpClient = new $.net.http.Client();    // load the library handling the OAuth protocol    var oAuthClientLib = $.import("sap.hana.xs.oAuth.lib", "oAuthClient");    // Google + API specific endpoints    var suffix = "/plus/v1/people/";    suffix += yourGoogleID;    // where we want to go    var request = new $.net.http.Request($.net.http.GET, suffix);    // initialize the HTTP destination and OAuth client    var myDestObj = $.net.http.readDestination("googleAPI", "googleAPIdestination");    var oaC = new oAuthClientLib.OAuthClient("googleAPI", "googleAPIdestination");    // SCOPES -> configure via XS Admin in the OAuth configuration package    //https://www.googleapis.com/auth/plus.me    // if you want to start from scratch, just set a breakpoint here and call this method    // oaC.userRevokeAccess();    // initialize the OAuth authorization code flow (and trace a debug message)    // do you know what happens if you call this URL via your Browser?    var url = oaC.userAuthorizeUri("https://sapgolf.hana.ondemand.com/googleAPI/callGooglePlusAPI.xsjs");    $.trace.debug("Auth URL is: " + url);    // if you called the URL via your browser paste the authorization code response into the 'code' variable (after uncommenting of course)    // var code;    // this is an alternative way to get the access tokens    // oaC.userGrantAccess(code);    // is our access token still valid, do we need to refresh it or didn't we receive anyone at all?    var validAT = oaC.hasValidAccessToken();    if (validAT) {        // call the API        response = oaC.processData(oHttpClient, request, myDestObj);    } else {        var validRT = oaC.hasValidRefreshToken();        if (validRT) {            var refreshStatus = oaC.userRefreshAccess();            if (refreshStatus === 200) {                // call the API                response = oaC.processData(oHttpClient, request, myDestObj);            }        } else {            $.response.setBody(JSON.stringify({                error: false,                errorDescription: "There are no tokens (access or refresh) for this application/user context available",                solution: "Authorize yourself via the following authorization URL",                authorizationURL: url            }));            return;        }    }    if (response) {        // display googles response        var myBody;        if (response.body)            try {                myBody = JSON.parse(response.body.asString());            } catch (e) {                myBody = response.body.asString();            }        $.response.contentType = "application/json";        $.response.status = 200;        $.response.setBody(JSON.stringify({            "status": response.status,            "body": myBody        }));    }
};
try {    callGoogle();
} catch (err) {    $.response.setBody("Failed to execute action: " + err.toString());
}

The first time you execute the application you will get something like:

The reason for this is the missing access token. We trigger the authorization flow by sending the required scopes, client id & secret and some more information to Google. Google sends us an authorization code back, which we finally exchange against access and refresh tokens. The access token will be stored in the OAuth secure store and used for upcoming API calls.

• click the provided authorizationURL

After you have approved the client request, you will see the well known Google OAuth log on UI

Log on to this one and grant access to the client

you will be redirected to the very same application you came from – this time showing you the JSON returned by the API call:

Conclusion

There are quite some configuration steps to carry out, before you can finally use OAuth authentication in you application. I hope you successfully completed all of them and most of all hope you see the potential you got at hand right now.

There will be some follow up posts dealing with the usage of the other pre delivered configuration packages, but also some going for even more advanced topics i.e. explaining how to create own configurations, how to debug outgoing requests and the like.

I would like to thank Michael Engler and Klaus Herter for their great support of this post!

I hope you all stay tuned and are looking forward the next posts.

Resources

Google

• https://console.developers.google.com/project
• https://developers.google.com/oauthplayground/
• https://developers.google.com/apis-explorer/#p/
• https://developers.google.com/+/api/latest/people/get
• https://developers.google.com/+/api/#auth
• https://developers.google.com/accounts/docs/OAuth2
• https://security.google.com/settings/security/permissions?hl=de&pli=1

SAP documentation

• http://help.sap.com/saphelp_hanaplatform/helpdata/en/90/878018cccd40f7a4b6754c04e2d34a/content.htm?frameset=/en/87/655028fc6b4ed8ac0b374a53782c85/frameset.htm&current_toc=/en/34/29fc63a1de4cd6876ea211dc86ee54/plain.htm&node_id=85&show_children=true#jump289

• Application configuration syntax
  • http://help.sap.com/saphelp_hanaplatform/helpdata/en/e5/b5d3a830ff4beea17c3efafe452065/content.htm?frameset=/en/87/655028fc6b4ed8ac0b374a53782c85/frameset.htm&current_toc=/en/34/29fc63a1de4cd6876ea211dc86ee54/plain.htm&node_id=86
• Client configuration syntax
  • http://help.sap.com/saphelp_hanaplatform/helpdata/en/4b/9d546faeff48f2b0c3cb33abd6daeb/content.htm?frameset=/en/e5/b5d3a830ff4beea17c3efafe452065/frameset.htm&current_toc=/en/34/29fc63a1de4cd6876ea211dc86ee54/plain.htm&node_id=87
• Client flavor syntax
  • http://help.sap.com/saphelp_hanaplatform/helpdata/en/67/57a4e3ed9f42bd86c5b5c3b264e090/content.htm?frameset=/en/4b/9d546faeff48f2b0c3cb33abd6daeb/frameset.htm&current_toc=/en/34/29fc63a1de4cd6876ea211dc86ee54/plain.htm&node_id=88
• HTTP destinations
  • http://help.sap.com/saphelp_hanaplatform/helpdata/en/ca/340c09551c40b7837e773b9d051821/content.htm?frameset=/en/87/655028fc6b4ed8ac0b374a53782c85/frameset.htm&current_toc=/en/34/29fc63a1de4cd6876ea211dc86ee54/plain.htm&node_id=79&show_children=false

Misc

• http://www.cheatography.com/kayalshri/cheat-sheets/oauth-end-points/
• https://en.wikipedia.org/wiki/OAuth
• http://oauth.net/