Intro

Hi everyone, welcome to the movie world again. In the previous blog post, we've used SAP HANA Info Access to build our smart app on desktop. With the simplicity of Info Access Dev Toolkit and its underneath text analysis feature in SAP HANA, it's possible for us to build such a powerful and fancy app in a short time. However, the UI framework of Info Access Dev Toolkit is more focused on the desktop instead of tablet or phone. When I used my iPhone to visit the app, the UI looked like the following.

Obviously the UI is not responsive as you can see, since it's for big screens. Usually we don't analyze so many things at the same time on our mobiles, as mobiles are for simple operations. It's crazy for you to show six charts and one list in only one screen on your mobile with drag and drop features. Do you like that? I think nobody likes. Since the scenario of our movie sentiment rating app is very simple, an idea came to my mind, how about build our app on mobile devices? And you can use it to pick up the best movie when you go to the cinema. It sounds cool. Let's do it together!

OData services

When we use SAP HANA Info Access, we just need an attribute view for the date layer. Since now we want to use SAPUI5 to build our movie app on mobile devices, first we need to expose few OData services for models in SAPUI5. Here I won't explain what OData is and how to use OData in SAP HANA XS, but you can find details from Home | OData - The Protocol for REST APIs and Data Access with OData in SAP HANA XS - SAP HANA Developer Guide - SAP Library

In our case, we want to create the following two OData services. (It seems the syntax highlighting still doesn't work. Sorry for the ugly formatting again. )

1. movies.xsodata

service namespace "movieRating.services" {
"movieRating.models::CV_MOVIES" as "CV_MOVIES"
key generate local "GenID"
parameters via entity "InputParams"
;}

The above OData service exposes the calculation view with input parameters described in Real-time sentiment rating of movies on SAP HANA (part 3) - text analysis and modeling. Since there are huge amounts of movies, we need to give two input parameters in order to get the limited results, one for start date and the other for end date. It will be used for displaying brief movie info including the movie title, poster, release date, rating and # of mentions. We can get the brief movie info in mention descending order with release date from 2014-11-11 to 2014-11-17 as follows.

2. tweets.xsodata

service namespace "movieRating.services" {
"movieRating.models::AT_TWEETS" as "AT_TWEETS"
with("movie_id", "user_screen_name", "user_profile_image_url", "sent", "created_at_str", "text", "id_counter")
key generate local "GenID"
;}

For this OData service, it's based on the attribute view in Real-time sentiment rating of movies on SAP HANA (part 3) - text analysis and modeling with limited attributes. We'll use it to show related mentions when we click a specific movie. For each mention, we want to show the user name, his/her profile image, the creation time of the tweet, the text and detected sentiment. We can also test the OData service to get the latest 20 mentions given a specific movie ID.

SAPUI5 - sap.m

Now let's use SAPUI5 to build the responsive UI for our movie app. So what is SAPUI5? SAPUI5 stands for SAP UI Development Toolkit for HTML5 and you can find details from SAPUI5 Developer Guide for SAP HANA - SAP Library. Since we want to build our movie app on mobile devices, we need to build views with sap.m library instead of sap.ui.commons library. So what is the difference between these two libraries? sap.ui.commons is for desktop/big screen and it's not responsive, meanwhile sap.m is for desktop/tablet/phone and it's responsive which means sap.m can adapt to the mobile OS, browser and screen size. You can also find more from this unit of "Next Steps in Software Development on SAP HANA" on openSAP. I just learned this unit today.

Structure

We decided to use sap.m to build our mobile app, but how can we start? First I highly recommend you to have a look at Model View Controller (MVC) - SAPUI5 Developer Guide for SAP HANA - SAP Library, since MVC is a very important concept in SAPUI5.

And you can also find a very detailed demo from An MVC Demo Application - SAPUI5 Developer Guide for SAP HANA - SAP Library. It's a perfect demo for MVC concept in SAPUI5 and the structure is similar with ours. The following image shows the structure of our mobile app.

• The "appView" only holds sap.m.App control to handle the page navigation between two pages "homeView" and "detailView".
• The "homeView" is displayed initially with two controls. sap.m.DateRangeSelection is used for date range selection, while sap.m.Carousel is used to show the brief movie info.
• The "detailView" will be showed when we click a movie in the carousel. It consists of several controls including sap.m.PullToRefresh and sap.m.List. With sap.m.PullToRefresh, we can pull to refresh sap.m.List to show the latest mentions. In addition, we also add the option "Load more mentions" which let the user to load more old mentions.

Implementation

index.html

<!DOCTYPE HTML>
<html>
  <head>
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta http-equiv='Content-Type' content='text/html;charset=UTF-8'/>
 
  <link rel="stylesheet" type="text/css" href="styles/style.css" />
 
 
  <script src="/sap/ui5/1/resources/sap-ui-core.js"
  id="sap-ui-bootstrap"
  data-sap-ui-libs="sap.m"
  data-sap-ui-theme="sap_bluecrystal">
  </script>
  <!-- only load the mobile lib "sap.m" and the "sap_bluecrystal" theme -->
 
 
  <script>
  sap.ui.localResources("movieratingmobile");
  var app = sap.ui.view({id:"appView", viewName:"movieratingmobile.app", type:sap.ui.core.mvc.ViewType.JS});
  app.placeAt("content");
  </script>
 
 
  </head>
  <body class="sapUiBody" role="application">
  <div id="content"></div>
  </body>
</html>

app.view.js

sap.ui.jsview("movieratingmobile.app", {
 
 
  /** Specifies the Controller belonging to this View.
  * In the case that it is not implemented, or that "null" is returned, this View does not have a Controller.
  * @memberOf movieratingmobile.app
  */
  getControllerName : function() {
  return "movieratingmobile.app";
  },
 
 
  /** Is initially called once after the Controller has been instantiated. It is the place where the UI is constructed.
  * Since the Controller is given to this method, its event handlers can be attached right away.
  * @memberOf movieratingmobile.app
  */
  createContent : function(oController) {
 
 
  var home = sap.ui.view({id:"homeView", viewName:"movieratingmobile.home", type:sap.ui.core.mvc.ViewType.JS});
  var detail = sap.ui.view({id:"detailView", viewName:"movieratingmobile.detail", type:sap.ui.core.mvc.ViewType.JS});
  return new sap.m.App("app").addPage(home).addPage(detail);
  }
 
 
});

app.controller.js

sap.ui.controller("movieratingmobile.app", {
 
 
/**
* Called when a controller is instantiated and its View controls (if available) are already created.
* Can be used to modify the View before it is displayed, to bind event handlers and do other one-time initialization.
* @memberOf movieratingmobile.app
*/
  onInit: function() {
 
  var date_to = new Date();
  var date_from = new Date(date_to.getTime() - 6 * 24 * 60 * 60 * 1000);
 
  var oView = this.getView();
  var oModel = new sap.ui.model.json.JSONModel("/movieRating/services/movies.xsodata/InputParams(date_from=datetime'" + this.dateString(date_from) + "',date_to=datetime'" + this.dateString(date_to) + "')/Results/?$format=json&$orderby=mention desc");
  oView.setModel(oModel);
 
  var dateRange = sap.ui.getCore().byId("dateRange");
  dateRange.setDateValue(date_from);
  dateRange.setSecondDateValue(date_to);
 
 
  this.app = sap.ui.getCore().byId("app");
 
  var oBus = sap.ui.getCore().getEventBus();
  oBus.subscribe("nav", "to", this.navToHandler, this);
  oBus.subscribe("nav", "back", this.navBackHandler, this);
  },
 
 
/**
* Similar to onAfterRendering, but this hook is invoked before the controller's View is re-rendered
* (NOT before the first rendering! onInit() is used for that one!).
* @memberOf movieratingmobile.app
*/
// onBeforeRendering: function() {
//
// },
 
 
/**
* Called when the View has been rendered (so its HTML is part of the document). Post-rendering manipulations of the HTML could be done here.
* This hook is the same one that SAPUI5 controls get after being rendered.
* @memberOf movieratingmobile.app
*/
// onAfterRendering: function() {
//
// },
 
 
/**
* Called when the Controller is destroyed. Use this one to free resources and finalize activities.
* @memberOf movieratingmobile.app
*/
// onExit: function() {
//
// }
 
  dateString: function(date) {
  var year = date.getFullYear();
  var month = date.getMonth() + 1;
  month = month > 9 ? month : "0" + month;
  var day = date.getDate();
  day = day > 9 ? day : "0" + day;
  return year + '-' + month + '-' + day;
  },
 
  navToHandler: function(channelId, eventId, data) {
 
 
  if (data && data.id) {
  // lazy load view
  if (this.app.getPage(data.id) === null) {
  jQuery.sap.log.info("now loading page '" + data.id + "'");
  this.app.addPage(sap.ui.jsview(data.id, "movieratingmobile." + data.id));
  }
 
 
  // Navigate to given page (include bindingContext)
  this.app.to(data.id, data.data.context);
  } else {
  jQuery.sap.log.error("nav-to event cannot be processed. Invalid data: " + data);
  }
  },
 
  navBackHandler: function() {
  this.app.back();
  }
});

home.view.js

sap.ui.jsview("movieratingmobile.home", {
 
 
  /** Specifies the Controller belonging to this View.
  * In the case that it is not implemented, or that "null" is returned, this View does not have a Controller.
  * @memberOf movieratingmobile.home
  */
  getControllerName : function() {
  return "movieratingmobile.home";
  },
 
 
  /** Is initially called once after the Controller has been instantiated. It is the place where the UI is constructed.
  * Since the Controller is given to this method, its event handlers can be attached right away.
  * @memberOf movieratingmobile.home
  */
  createContent : function(oController) {
 
  var poster = new sap.m.Image({
  densityAware: false,
  src: {
  path: "poster",
  formatter: function(fValue) {
  return fValue.replace(/tmb/, "det");
  }
  },
  press: [oController.showDetail, oController]
  });
 
  var title = new sap.m.Text({
  text: "{title}"
  }).addStyleClass("title");
 
  var release_date = new sap.m.Text({
  text: "{release_date}"
  });
 
  var rating = new sap.m.RatingIndicator({
  iconSize: "12px",
  value: {
  path: "rating",
  formatter: function(fValue) {
  return parseFloat(fValue) / 2;
  }
  }
  });
 
  var score = new sap.m.Text({
  text: {
  path: "rating",
  formatter: function(fValue) {
  return " " + fValue;
  }
  }
  }).addStyleClass("score");
 
  var rating_score = new sap.m.HBox({
  items: [rating, score]
  });
 
  var mention = new sap.m.Text({
  text: {
  path: "mention",
  formatter: function(fValue) {
  return "(" + fValue + " Mentions)";
  }
  }
  });
 
  var vbox = new sap.m.VBox({
  alignItems: sap.m.FlexAlignItems.Center,
         items: [poster, title, release_date, rating_score, mention]
  });
 
  var carousel = new sap.m.Carousel({
  loop: true,
  showPageIndicator: false,
  pages: {
  path: "/d/results",
  template: vbox
  }
  });
 
  var dateRange = new sap.m.DateRangeSelection("dateRange", {
  change: [oController.changeDate, oController]
  });
 
  var page = new sap.m.Page({
  title: "Movie Sentiment Rating",
  content: [dateRange, carousel]
  });
 
  return page;
  }
});

home.controller.js

sap.ui.controller("movieratingmobile.home", {
 
 
/**
* Called when a controller is instantiated and its View controls (if available) are already created.
* Can be used to modify the View before it is displayed, to bind event handlers and do other one-time initialization.
* @memberOf movieratingmobile.home
*/
// onInit: function() {
//
// },
 
 
/**
* Similar to onAfterRendering, but this hook is invoked before the controller's View is re-rendered
* (NOT before the first rendering! onInit() is used for that one!).
* @memberOf movieratingmobile.home
*/
// onBeforeRendering: function() {
//
// },
 
 
/**
* Called when the View has been rendered (so its HTML is part of the document). Post-rendering manipulations of the HTML could be done here.
* This hook is the same one that SAPUI5 controls get after being rendered.
* @memberOf movieratingmobile.home
*/
// onAfterRendering: function() {
//
// },
 
 
/**
* Called when the Controller is destroyed. Use this one to free resources and finalize activities.
* @memberOf movieratingmobile.home
*/
// onExit: function() {
//
// }
 
  changeDate: function(oEvent) {
  this.getView().getModel().loadData("/movieRating/services/movies.xsodata/InputParams(date_from=datetime'" + sap.ui.controller("movieratingmobile.app").dateString(oEvent.getParameters().from) + "',date_to=datetime'" + sap.ui.controller("movieratingmobile.app").dateString(oEvent.getParameters().to) + "')/Results/?$format=json&$orderby=mention desc");
  },
 
  showDetail: function(oEvent) {
  var oBindingContext = oEvent.getSource().getParent().getBindingContext();
 
 
  var oBus = sap.ui.getCore().getEventBus();
  oBus.publish("nav", "to", {
  id: "detail",
  data: {
  context : oBindingContext
  }
  });
  }
});

detail.view.js

sap.ui.jsview("movieratingmobile.detail", {
 
 
  /** Specifies the Controller belonging to this View.
  * In the case that it is not implemented, or that "null" is returned, this View does not have a Controller.
  * @memberOf movieratingmobile.detail
  */
  getControllerName : function() {
  return "movieratingmobile.detail";
  },
 
 
  /** Is initially called once after the Controller has been instantiated. It is the place where the UI is constructed.
  * Since the Controller is given to this method, its event handlers can be attached right away.
  * @memberOf movieratingmobile.detail
  */
  createContent : function(oController) {
 
  var pullToRefresh = new sap.m.PullToRefresh({
  refresh: [oController.refreshList, oController]
  });
 
  var poster = new sap.m.Image({
  densityAware: false,
  src: "{poster}"
  });
 
  var title = new sap.m.Text({
  text: "{title}"
  }).addStyleClass("title");
 
  var release_date = new sap.m.Text({
  text: "{release_date}"
  });
 
  var rating = new sap.m.RatingIndicator({
  iconSize: "12px",
  value: {
  path: "rating",
  formatter: function(fValue) {
  return parseFloat(fValue) / 2;
  }
  }
  });
 
  var score = new sap.m.Text({
  text: {
  path: "rating",
  formatter: function(fValue) {
  return " " + fValue;
  }
  }
  });
 
  var rating_score = new sap.m.HBox({
  items: [rating, score]
  });
 
  var mention = new sap.m.Text({
  text: {
  path: "mention",
  formatter: function(fValue) {
  return "(" + fValue + " Mentions)";
  }
  }
  });
 
  var vbox = new sap.m.VBox({
         items: [title, release_date, rating_score, mention]
  }).addStyleClass("info");
 
  var hbox = new sap.m.HBox({
  items: [poster, vbox]
  }).addStyleClass("movieInfo");
 
  var oBar = new sap.m.Toolbar({
  height: "90px",
  content: [hbox]
  });
 
  var listItem = new sap.m.FeedListItem({
  iconDensityAware: false,
  sender: "{user_screen_name}",
  icon: "{user_profile_image_url}",
  info: "{sent}",
  timestamp: "{created_at_str}",
  text: "{text}"
  });
 
  var oList = new sap.m.List({
  items: {
  path: "/d/results",
  template: listItem
  }
  });
  oList.setModel(model);
 
  var loader = new sap.m.StandardListItem({
  type: sap.m.ListType.Active,
  title: "Load 20 more mentions...",
  press: [oController.loadMore, oController]
  });
 
  var page = new sap.m.Page({
  title: "Movie Detail",
  showNavButton: true,
  subHeader: oBar,
  content: [pullToRefresh, oList, loader],
  navButtonPress: [oController.showHome, oController]
  });
 
  this.addEventDelegate({
  onBeforeShow: function(evt) {
  this.setBindingContext(evt.data);
  oController.initializeList(evt);
  }
  }, this);
 
  return page;
  }
 
 
});

detail.controller.js

var movie_id, max_id_counter;
var model = new sap.ui.model.json.JSONModel();
 
 
sap.ui.controller("movieratingmobile.detail", {
 
 
/**
* Called when a controller is instantiated and its View controls (if available) are already created.
* Can be used to modify the View before it is displayed, to bind event handlers and do other one-time initialization.
* @memberOf movieratingmobile.detail
*/
// onInit: function() {
//
// },
 
 
/**
* Similar to onAfterRendering, but this hook is invoked before the controller's View is re-rendered
* (NOT before the first rendering! onInit() is used for that one!).
* @memberOf movieratingmobile.detail
*/
// onBeforeRendering: function() {
//
// },
 
 
/**
* Called when the View has been rendered (so its HTML is part of the document). Post-rendering manipulations of the HTML could be done here.
* This hook is the same one that SAPUI5 controls get after being rendered.
* @memberOf movieratingmobile.detail
*/
// onAfterRendering: function() {
//
// },
 
 
/**
* Called when the Controller is destroyed. Use this one to free resources and finalize activities.
* @memberOf movieratingmobile.detail
*/
// onExit: function() {
//
// }
 
 
  showHome: function() {
  var oBus = sap.ui.getCore().getEventBus();
  oBus.publish("nav", "back");
  },
 
  initializeList: function(oEvent) {
  movie_id = new sap.ui.model.json.JSONModel(oEvent.data).getProperty("/oModel/oData" + oEvent.data.sPath + "/id");
  model.loadData("/movieRating/services/tweets.xsodata/AT_TWEETS/?$format=json&$filter=movie_id eq " + movie_id + "&$top=20&$orderby=id_counter desc", null, false);
  this.updateMax(model);
  this.getView().getContent()[0].getContent()[0].setDescription("Last updated time: " + new Date().toUTCString());
  },
 
  refreshList: function(oEvent) {
  model.loadData("/movieRating/services/tweets.xsodata/AT_TWEETS/?$format=json&$filter=movie_id eq " + movie_id + "&$top=20&$orderby=id_counter desc", null, false);
  this.updateMax(model);
  this.getView().getContent()[0].getContent()[0].setDescription("Last updated time: " + new Date().toUTCString()).hide();
  },
 
  loadMore: function() {
  var mentionModel = new sap.ui.model.json.JSONModel();
  mentionModel.loadData("/movieRating/services/tweets.xsodata/AT_TWEETS/?$format=json&$filter=movie_id eq " + movie_id + " and id_counter lt '" + max_id_counter + "'&$top=20&$orderby=id_counter desc", null, false);
  var mentions = this.updateMax(mentionModel);
  for (var i in mentions) {
  this.getView().getContent()[0].getContent()[1].addItem(new sap.m.FeedListItem({
  iconDensityAware: false,
  sender: mentions[i].user_screen_name,
  icon: mentions[i].user_profile_image_url,
  info: mentions[i].sent,
  timestamp: mentions[i].created_at_str,
  text: mentions[i].text
  }));
  }
  },
 
  updateMax: function(model) {
  var mentions = model.getProperty("/d/results");
  max_id_counter = mentions[mentions.length - 1].id_counter;
  return mentions;
  }
});

Look & feel

1. Homepage displaying new release movies info (e.g., rating and # of mentions) within carousel in the last seven days

2. Swipe the carousel to see various movies

3. Click a movie to see the latest 20 tweet mentions/sentiments of this movie

4. Scroll down/up to see mentions

5. Pull to refresh the latest 20 mentions

6. You can always load 20 more old mentions

7. Select the release date range to get movies you want to see

Movie Sentiment Rating using sap.m

Similar with Real-time sentiment rating of movies on SAP HANA (part 4) - SAP HANA info access, I've also recorded a video for you. Have fun.

Next steps

In this blog post, we've learned how to use SAPUI5, especially the sap.m library and we've built our movie sentiment rating app with sap.m. So till now, we've completely developed two smart apps, one based on SINA in previous blog and the other using sap.m which is described in this blog. In the next blog, let's have an open discussion. I'll show you the source code on GitHub and some future work.

Hope you enjoyed the development journey and picking up your favorite movie on your phone.

HANA is all-sufficient as a platform. It has everything out of the box to develop applications - DB, XS Engine, SAPUI5.

Even if HANA mostly considered for enterprise application's development it is also can be used as a box for public web applications/sites as well.

In this text below  I'm going to talk about some points to have in mind when you start doing non-enterprise systems on HANA.

So what is usual web application in terms of user access?

1. everybody can access it
2. soon or later you will be forced to login into application somehow to get the service
3. login via social networks is widely used and can be considered as de-facto standard

Lets look at what is HANA proposes in user access area. Currently HANA supports following authentication methods:

For non-interactive logon:

1. X509 authentication
2. SPNego
3. SAP logon ticket
4. Basic authentication

For interactive logon:

1. SAML
2. Form-based authentication

And 'None' for public applications.

This is one more proof for the statement 'HANA is still mostly enterprise-oriented'

The only thing we can take from the above list for out use case is ability to make public applications. All the rest must be done manually.

Custom session management (login step) can look like:

I think it is clear that during any next call from UI to backend sessionId must be provided as a cookie and validated on HANA server.

This approach is nice but there is one issue here - sessionId generation.

Lets think how we can do that. Actually there are two ways: do it on our own or try to reuse token/cookies from social network.

Expanding first way we can:

1. on client side:
   1. Math.random()
   2. jQuery.sap.uid()
   3. other jQuery/JS plugins, libs
2. on XS Engine side:
   1. same as in #1 except jQuery(unless you ported jQuery on XS )
3. on HANA side:
   1. SELECT SYSUUID FROM DUMMY;
   2. SELECT RAND() FROM DUMMY;
   3. some magic like: populate table with 'random' values via #1 or 2 and 'randomly' select from this table

With all of these points there is one huge issue - they are not designed to be used as session Id generators. They are producing values with low entropy.

This contradicts to basic requirements for session id generation (some details can be found here).

Moving to second way. Reusing token/cookie (provided by social network) directly is not really good idea as well. Because for example it can change over the time (Facebook can validate current FB cookie and issue new one still pointing to the same user's session). All we can do here is add some salt and apply hash function. We could if were provided with api by HANA team

So there is no direct, obvious way to generate reliable, strong sessionId from HANA applications code. But there is tricky workaround as usual

Approach is the following:

1. create destination pointing to localhost
2. create empty password-protected xsjs service
3. create technical user with access to package with password-protected service
4. call protected service via destination passing userName/password
5. extract sessionId (and csrf protection token if needed) from response cookies/headers

As simple as that

Simplest implementation can look like:

utils.xsjs listing:

function generateSessionIdAndToken() {
 
    var destination = $.net.http.readDestination("training.dlapanik.SessionId", "localhost");
    var client = new $.net.http.Client();
    var request = new $.net.http.Request($.net.http.GET, "/training/dlapanik/SessionId/private/getSessionId.xsjs");
    request.headers.set("Authorization", "Basic U2Vzc0lkVGVjaFVzZXI6SW5pdGlhbDIzNA===");
    request.headers.set("X-CSRF-Token", "Fetch");
 
    client.request(request, destination);
    var response = client.getResponse();
 
    if (response.status === $.net.http.OK) {
        $.response.status = $.net.http.OK;
        return {
            sessionId : response.cookies.get("xsSessionId"),
            csrfToken : response.headers.get("x-csrf-token")
        };    
    } else {
        return {
            status : "not OK:" + response.status
        };
    }
}

Here we see pure XSJS Outbound API usage.

There only question can occur is about header with name 'Authorization'. Actually this is the way to pass user name/password into the service call. The value you see after 'Base' is base64 function from user/pass combination. For example for SessIdTechUser:Initial234 you will get value U2Vzc0lkVGVjaFVzZXI6SW5pdGlhbDIzNA==. I used Base64 Decode and Encode - Online to create authorization token providing <user_name>:<password> as input.

Full project code can be found on github. Please note that example will work in case you activate it into 'training.dlapanik' package. In case package is diferent please adjust the code.

All in one slide:

To test launch http://<host>:<port>/training/dlapanik/SessionId/utils.xsjs, will get json object like

So summarize: in this article we saw how to 'ask Hana' to generate sessinoId value for public non-enterprise HANA applications.

Hello All,

In this blog, I have explained about how to insert records into SAP HANA from a reporting tool like Web Intelligence. This concept can be used in most reporting tools like Dashboards, Lumira & Design Studio to insert data into SAP HANA.

Please find below steps to acheive this.

1. Create table in sap HANA

Create column table with name T_B_SALES with below structure in SAP HANA.

2. Create SAP HANA Repository

Goto Window -> Show View from Top Menu Bar

Select SAP HANA Repositories.

Right click on the Repository Area & create new repository.

Give Repository Name and click finish.

You will see the Repository successfully created in SAP HANA.

  3. reate XS Application

Go to Window -> Show View from Top Menu Bar and select Project Explorer.

4. In Project Explorer Create New XS Project

Right click in Project Explorer Area New-> Project ->SAP HANA Development->XS Project.

Give Project Name & Click on Finish. Now you will see a XS Project folder created in Project Explorer.

5. Create .xsapp & .xsaccess files in XS Project

Right Click on XS Project folder and select ‘file’ option.

  Give file name as .xsapp. Keep this file blank. This file is for XS engine to understand there is an XS application in this package.

Similarly create .xsaccess file & write below code in it.

Right Click on our XS Project XS_BOOK_STORE and create XS JavaScript Source File.

Name it as “insert_purchase_record.xsjs” and write below code in it.

  var cid = $.request.parameters.get('cid');

  var bname = $.request.parameters.get('bname');

var no_of_copies = $.request.parameters.get('no_of_copies');   
$.response.headers.set('access-control-allow-origin','*');

  varconn = $.db.getConnection();

   var pstmt = conn.prepareStatement("INSERT INTO \"UNAME\".\"T_B_SALES\"
values ('"+cid+"','"+bname+"',current_date,"+no_of_copies+")");

var rs = pstmt.execute();

   $.response.setBody( "Entry has been created Successfully.");

  }

 

  conn.commit(); 

  conn.close();

Note: UNAME is username in SAP HANA

Share Project

Right Click On Project and goto Team-> Share Project

Once your Project is shared small Yellow cylinder will appear at Project Logo.

Now Select all the 3 files (.xsapp,.xsaccess and insert_purchase_record.xsjs) and Activate the files.

Then all files will show the small Yellow cylinder at file logo after successful activation of files.

6. Create a WEBI report which will write data back to SAP HANA.

Create WEBI report with No data Source. As we do not want to see any data, and need to insert data into HANA DB.

Create 3 Variables in BO as below

Define 3 input control Text Boxes for each of variables.

Create Report with above variables as below. Try to articulate it to appeal to users.

Do not forget to select cell property of insert cell to Hyperlink, as below.

Now, right click On the “Insert” Text, Goto Linking->Hyperlink and put below code in window & parse it.

http://<HANA_SERVER>:<PORT>/XS_BOOK_STORE/insert_purchase_record.xsjs?cid=B0000001&bname=INDIA&no_of_copies=1 

  PORT: Mostly = 8000 (for HANA XS ENGINE)

Click on parse. Below window will now appear,

Now assign the BOBJ variables to URL parameters, as below.

7. Execute the BO Report

Now let’s test our XS application..!

Click on insert, this will take you to the browser and will prompt you to enter HANA password.

  After entering the password and clicking OK it you show you following window.

Now Check HANA table T_B_SALES. You will see one entry created in table..!!

You can now also try to update and delete the records in HANA DB. You can use the same table, on which we are performing DML operations, as the BOBJ report source and can see the records gets change at WEBI Report level at runtime.

Hope you find this worth of reading..!! Thanks

Hi,

I have submitted an idea on SAP Idea Incubator on custom Stock In Transit Report :

The main idea behind the report is to help the purchasing managers get a detailed view of the status of stock in transit in a way which will be useful to them. The current SAP function MB5T does not provide key information to buyers as they need it. The report will help the buyers know exactly in what status their order is currently. There can be more functionality built in to the report.

If some one is interested in developing this idea, I will be more than happy to provide him with more inputs.

Here is the link with more details of the idea.

https://ideas.sap.com/SAPHANAIdeaIncubator/stock-in-transit-report-for-intercompany?current_tab=Recent&row_num=1&getparameters=1

Eagerly awaiting your feedback.

Regards,

Amit

Wrap-up

Hello and welcome back to the last blog post of movie sentiment rating series. Now it's time to recap what we have done and discuss some future work. First of all I wanna share the series and project with you as I promised in the first blog. Here you go.

Series

Real-time sentiment rating of movies on SAP HANA (part 1) - create schema and tables

Real-time sentiment rating of movies on SAP HANA (part 2) - data preparation

Real-time sentiment rating of movies on SAP HANA (part 3) - text analysis and modeling

Real-time sentiment rating of movies on SAP HANA (part 4) - SAP HANA info access

Real-time sentiment rating of movies on SAP HANA (part 5) - OData services and SAPUI5

Real-time sentiment rating of movies on SAP HANA (part 6) - wrap-up and looking ahead

Project

linkinchow/movie · GitHub

After you import the XS project, the project structure should look similar with mine. For better understanding, I made some notes on the structure and you can find these design-time objects in corresponding blog posts easily.

OK. Now let's take a look at what we've done. We've rebuilt the movie sentiment rating app using pure SAP HANA XS and text analysis. And now we have two real-time apps using the same dataset, one for desktop based on SINA and the other for mobile using sap.m. During the implementation, we've used some new XS features released in recent SPSs such as CDS and outbound connectivity in SPS06, job scheduling in SPS07. These features did not exist when I built the first version of this smart app in May 2013. With the rapid development of SAP HANA XS, we can always improve our SAP HANA native apps. Then let's have a quick look at these two apps respectively.

Movie sentiment rating based on SINA

Major features:

- Search everything in real-time. With the full text index, you can search movie titles, release dates, studios, sentiments, tokens and so on. For example, you can search tweets about a specific movie in this evening from iPhone.

- Analyze everything in real-time. You can analyze how many people like/dislike a certain movie or the overall sentiment of movies in a certain studio. You can even track the sentiment trending. There are lots of things you can analyze.

Movie sentiment rating using sap.m

Major features:

- For movie goers, you can see the sentiment rating of new release movies based on tweets analysis and select your favorite movie in real-time. From # of mentions you can also infer the popularity. And you can even select the release date range to check more.

- You can jump into details to see what people are now talking about this movie.

Compared with Rotten Tomatoes?

Out of curiosity I compared the rating of top 10 popular new release movies this week in my app with Rotten Tomatoes. The following result is based on 2014-11-23 04:00:00 UTC.

Real real-time?

I watched The Hunger Games: Mockingjay - Part 1 today and have just posted a tweet. You see both apps show my tweet immediately since with the combination of XSJS outbound connectivity and job scheduling, we can crawl tweets in real-time!

Looking ahead

Now let's jump into the looking ahead part and discuss some future work about the movie sentiment rating app. There are still lots of things we can improve for this app. I just listed several points as follows.

1. Genres, cast and directors

Do you remember in the first blog, besides the basic movie metadata, we also have three additional tables "Genres", "AbridgedCast" and "AbridgedDirectors"? Yeah, in the second blog, we searched and inserted data into these tables as well. So why not display genres, cast and directors in the UI? In order to show this info, we can Create Hierarchies in an Attribute View.

2. Geospatial analysis

The "Tweets" table has two columns which we did not use in our app, i.e., longitude and latitude. Actually we search the location info of tweets in our smart app and if the user provides the info we will store it. You can find the logic in the second blog. With the location info of tweets, we can make some interesting geospatial analysis, e.g., we can analyze the sentiment of movies in each state or for a certain movie/genre, we can compare the sentiment between eastern US and western US. Moreover, if we have age and gender info, we can analyze something more interesting.

3. Search more tweets for popular movies

Because of API Rate Limits | Twitter Developers, it's impossible for us to search all related tweets. So, the current approach is searching tweets about new release movies every two minutes. There is no difference between movies and each movie is equal to handle. Now the problem comes, for popular movies maybe there are thousands of tweets in just two minutes which we cannot get them once and we will miss some. On the other hand, for some unpopular movies, maybe there is no related tweet in an hour, so we do not need to search them so frequently. It's a waste if we still search them every two minutes. I think it would be better if we can create a dynamic mechanism which can search tweets for popular movies more frequently and tweets for unpopular movies less frequently.

4. Average sentiment of tweets

Currently if a tweet contains several sentiments, they will be regarded as several mentions. So, sometimes you will see several consequent mentions with the same username and the same time. Actually they belong to the same tweet. In order to avoid this, we can create an average sentiment for each tweet, something like 1 positive + 1 negative = 1 neutral...

5. Local time

Now the movie app uses UTC time everywhere, but it's not a good user experience, especially for mobile app. You are more willing to use your local time. So, this is also a feature which we can improve.

Further resources about the movie app

Blogs

Real-time sentiment rating of movies on SAP HANA One

#movienight on SAP HANA One

Videos

Webinars

Big Data - Sentiment analysis of movies on SAP HANA One | BrightTALK

Movie Sentiment Rating App Powered by Text Analysis on HANA One | BrightTALK

That's it! Hope you enjoyed the movie sentiment rating app the same way we did. Why not use it to pick up your favorite movie? Have fun!

ELIMINATING CURSORS IN HANA

INTRODUCTION:

Use of cursors in HANA degrades the performance a lot as it performs operation on single record at a time due to which the read and write on table happens more number of times. An alternate to this is to fetch all the records at one stretch and store it in a temporary table type and perform the calculation together. I would like to share a few complex use cases where we eliminated the use of cursors.

USE CASE 1:Reads data by fetching the most relevant record and does update of the most relevant record based on the condition

PROBLEM STATEMENT: We need to select candidates for air Force. There are 2 levels of exams where a single candidate has 2 attempts to clear each level.

Conditions are:

     1.  There are 3 measure based on which a candidate is tagged ideal for that level

          Level 1: Marks  >= 70, Age >= 25, weight >= 70

          Level 2: Mark >= 90, Age >= 30, weight >= 75 

          The order of importance of the measures is (in descending order):

1. Marks
2. Age
3. Weight

     2.   If the candidate has passed level 2 then his job title should be updated as ‘Pilot’ and should not be considered in level 1 selection. And if he has not           passed then check if he has passed in level 1, if so then update his job title as ‘Co-Pilot’.

PERFORMANCE IMPROVEMENTS:

Before :

After:

Table :1 MEASURE  [row-count :1,00,000 approx.]

Table 2 : IDEAL

Table 3 : ELIGIBILITY

Table 4: JOB_TABLE[row-count :1,00,000 approx.]

Bottle necks:

• If the candidate has already been tagged as ‘Pilot’ in category B then he shouldn’t be considered for evaluation in Level 1 which can be done using a NOT IN check but it again degrades the performance.
• The data should come in the sequence with the highest relevant score of the candidate getting updated for that particular level.    

------CURSOR ELIMINATED     

-- TEMPORARY TABLE VARIABLE THAT HAS ALL THE ELIGIBLE CANDIDATES FOR LEVEL 1 AND LEVEL 2

-- TEMPORARY TABLE VARIABLE THAT HAS ALL THE ELIGIBLE CANDIDATES FOR LEVEL 2

-- WE DO A OUTER JOIN TO AVOID SELECTION OF CANDIDATE WHO CLEARED LEVEL 2

-- SELECT ONLY THE RECORDS WHERE 'DEL' FLAG IS NULL SO THAT WE SELECT THE CANDIDATES WHO HAS NOT CLEARED LEVEL 2 BUT ELIGIBLE FOR EVALUATION IN LEVEL 1

CONCLUSION:

• Assign cursor s to table variables and then process the logic of the cursor at once.
• Cartesian product as a result of inner Join can be used to get all the records that are processed through nested loop.
• We can eliminate NOT IN conditions by setting a ‘DEL’ flag in table A and then do a left outer join with table B and then select the only records having flag as NULL which ensures the records are not selected again in table B this method improves the performance drastically when you are performing a NOT IN condition on a large data set.

An tablet application which will be in contact with the main HANA server and give real-time updates (charts + notifications) to the business owner/representative or concerned party so that the response time for the situation can be minimized & real-time analysis can be viewed.

Platforms:

iOS

Android

Windows

What do you think? Please add in the comments.

Idea Link - SAP HANA mobility : View Idea

Doing a startup is a risky business. According to a study by Mashable, 75% of all startups and 90% of all products fail. At SAP Startup Focus, we are in the business of helping startups increase those odds considerably. We do this by providing them an amazing software platform SAP HANA they can build their solutions on and by helping them with the go-to-market side of things. Currently we have over 1,700 startups that are building solutions on our HANA platform, from more than 55 countries, with 130 validated solutions available for sale to customers.

As compared to a few years ago, it has indeed become a lot easier and a lot cheaper to build an application you can take to market, or shop around to investors. With widespread adoption of the Cloud and increasing usage of Mobile as a consumption mechanism, even scrappy startups can now develop sophisticated products without getting a 2^nd mortgage on the house.

And that is where our very valued partners at SUSE come in. They understand unequivocally that for startups to succeed in a brutally competitive global environment, they not only need access to cutting edge technology like SAP HANA but also need it in a manner that is uniquely suited to their geographic and financial situation. For e.g. in several emerging markets access to commercial-grade bandwidth remains an ongoing issue which means that developing on a platform that is only available in the Cloud remains a logistical challenge.

Hence, we are very proud to announce that starting right now qualified startups in the SAP Startup Focus program will be eligible to receive for a 6-month period a comprehensive developer version of SAP HANA on SUSE Enterprise Linux Server as a single, downloadable Virtual Machine (VM). A single VM reduces the barriers to adoption for the SAP HANA platform, and will allow for startups to quickly evaluate and rapidly prototype solutions while running a developer version of SAP HANA on their local machine.  This will significantly reduce cost, and increase the agility of a startup in the initial product development phase of their lifecycle.

Additionally, startups will also receive free training via SUSE’s Certified Linux Administrator (CLA) Academy, a $2,375 USD value. Pre-sales electronic support and membership in SUSE’s partner program operated by PartnerNet®, which provides access to additional SUSE software, is also included.

To download the VM, please visit the SAP HANA Marketplace, the destination for startups to discover, try and buy the SAP HANA platform as well as applications based on SAP HANA, located at http://marketplace.saphana.com/p/3377

SLT
Concept-trigger based.

DB triggers
controls delta mechanism involved in it.

Logging
table records the data record last loaded and this helps with delta working
properly during SLT.

Read module
pushes data in to replication server,

If any
transformation needed it will be handled by transformation engine and write
engines pushes that to application tables in SAP HANA.

Hope this
helps understand the Trigger based approach

SAP HANA Studio for MAC = eclipse + SAP plugins

You have probably landed here because you could not find the HANA Studio installation files for the OS X platform which has mysteriously disappeared from the SAP HANA download website. Surprise, surprise...

Until it gets listed again, another way to run HANA Studio on Mac is to install SAP HANA Tools. I tried it out myself and found some minor issues that I will highlight here. Installation should be a quick and simple experience. After all, HANA Studio is basically eclipse + some plugins (i.e. SAP HANA Tools).

Below, I have listed the steps to follow in order to get HANA Studio up and running on Mac running Yosemite (OS X 10.10.1).

Step 1: Prerequisites

The JAVA runtime environment is required to run eclipse so you will have to install it first. The question is which one? On the SAP HANA Tools website, it is recommended to use JRE version 1.6+.  For Mac users, Apple supplies their own version of Java for Java versions 6 and below. If prompted by the OS, do not install JAVA for OS X (http://support.apple.com/kb/dl1572) as it is not compatible with eclipse Luna. Personally, I have a preference for using the latest version of JAVA (security) and so I will be installing JAVA SE 8u25 (I am going with the JDK since I need it for other software development purposes as well).

• Download the JDK from the following link: http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html (MAC OS X x64)
• Install it using the installer

Step 2: Installing eclipse

The next step is to install the eclipse IDE.

• Download eclipse standard 4.4 version for MAC OS X (Cocoa) 64-bit version using the following link: http://www.eclipse.org/downloads/packages/release/Luna/R
• Simply unpack it to your hard drive
• Once unzipped, copy (or drag n drop) the eclipse folder into your Applications folder

Step 3: Installing SAP HANA Tools

Third and final step, you will need to install SAP HANA Tools in eclipse. This is where I encountered minor issues that were not expected...

• Open Eclipse IDE
• Select a workspace at the prompt
• Click on Help> Install New Software

• At this point, I got a Missing requirement AFL connector error while trying to install the HANA Tools. To resolve this,  I had to install the Eclipse SDK plugin.
• In the dropdown menu, select "The Eclipse Project Updates" or add it if you do not see it "http://download.eclipse.org/eclipse/updates/4.4"
• Make sure the checkbox is checked for "Contact all update sites..."
• Select Eclipse SDK.
• Click Next, agree to the license agreement then Finish

• SAP Network users: you might encounter a Repository not found error like I did if you are on the SAP-Corporate network as access to the repository is blocked. Therefore you will have to switch to another network (i.e. SAP-Internet) and repeat the previous steps if needed.
• If successful, you will be prompt to restart eclipse
• Open eclipse again
• Select a workspace at the prompt
• Click on Help> Install New Software
• Add the following repository: https://tools.hana.ondemand.com/luna
• Make sure the checkbox is checked for "Contact all update sites..."
• Select SAP HANA Tools. If you did not install the eclipse SDK plugin in the previous step, you might encounter Missing requirement AFL connector error here like I did.
• Click Next, agree to the license agreement then Finish

You will be prompted to re-start eclipse which means that the installation process is successfully completed.

Now, you can simply connect to a HANA system and get started with the fun part of development.

Detect and Act: Insight from Event Streams

SAP HANA smart data streaming allows you to extract insight from real-time information streams and respond immediately!

SAP HANA smart data streaming lets you capture, analyze, and act on the continuous flow of new information that streams into your business, identify emerging threats and opportunities as they happen, and respond immediately. SAP HANA smart data streaming is a highly scalable event processor for the SAP HANA platform that enables you to capture and process streams of events from many sources in real-time to provide sub-second response to incoming data.

• Visit the smart  data streaming playlist on the SAP HANA Academy
• Visit Smart Data Streaming Developer Center

Transform your Business: Manage all your data cost effectively with the performance you demand!

SAP HANA dynamic tiering allows you to extend HANA memory with disk-backed column store technology.

SAP HANA dynamic tiering is a highly scalable, integrated option that gives application developers centralized operational control to cost effectively manage very large data sets – terabytes to petabytes – allowing uses to characterize data into temperature tiers and move the data from one temperature tier to another within the same database. With SAP HANA dynamic tiering you can transform your business by managing all your data cost effectively with the performance you demand.

• Visit the dynamic  tiering video tutorial pagein the SAP HANA SCN Developer site for great overview and installation videos

For more information on the HANA SPS09 release read the blog by Mike Eacrett on “What’s New for HANA SPS09”

Prior to Hana SP9 SAP suggested to use different tools to get data into Hana: Data Services (DS), System Landscape Transformation (SLT), Smart Data Access (SDA), Sybase Replication Server (SRS), Hana Cloud Integration - DS (HCI-DS),... to name the most important ones. You used Data Services for batch transformations of virtually any sources, SLT for realtime replication of a few supported databases with little to no transformations, HCI-DS when it comes to copying database tables into the cloud etc.

With the Hana Smart Data Integration feature you get all in one package plus any combination.

The user however has very simple requirements when it comes to data movement these days:

• Support batch and realtime for all sources
• Allow transformations on batch and realtime data
• There should be no difference between loading local on-premise data and loading over the Internet into a cloud target other than the protocol being used
• Provide one connectivity that supports all
• Provide one UI that supports all

The individual tools like Data Services do make sense still for all those cases the requirement matches the tool's sweet spot. For example a customer not running Hana or where Hana is just yet another database, such a user will prefer a best of breed standalone product like Data Services always. Customers requiring to merge two SAP ERP company codes will use SLT for that, it is built for this use case. All of these tools will continue to be enhanced as standalone products. In fact this is the larger and hence more important market! But to get data into Hana and to use the Hana options, that is when it becomes hard to argue why multiple external tools should be used, each with its own connectivity and capability.

In addition to that the Hana SDI feature tries to bring the entire user experience and effectiveness to the next level, or lays the groundwork for that at least.

Designing Transformations

Let's start with a very simple dataflow, I want to read news from CNN, check if the text "SAP" is part of the news description and put the result into a target table. Using Hana Studio, we create a new Flowgraph Model repo object and I dragged in the source, a first simple transformation and the target table. Then everything is configured and can be executed. So far nothing special, you would do the same thing with any other ETL tool.

But now I want to deal with the changes. With any ETL tool in the market today, I would need to build another dataflow handling changes for the source table. Possibly even multiple in case deletes have to be processed differently. And how do I identify the changed data actually?

With Smart Data Integration all I do in above dataflow is to check the realtime flag, everything else happens automatically.

How are changes detected? They are sent in realtime by the adapter.

What logic needs to be applied on the change data in order to get it merged into the target table? The same way as the initial load did, considering the change type (insert/update/delete) and its impact on the target.

The latter is very complex of course, but we when looking at what kind of dataflows the users have designed for that, we were able to come up with algorithms for each transformation.

The complexity of what happens under the cover is quite huge, but that is the point. Why should I do that for each table when it can be automated for most cases? Even if it works for 70% of the cases only, that is already a huge time saver.

Ain't that smart?

The one thing we have not been able to implement in SP9 is joins, but that was just a matter of development time. The algorithms exists already and will be implemented next.

Adapters

How does Hana get the news information from CNN? Via a Java adapter. That is the second major enhancement we built for SP9. Every Java developer can now extend Hana by writing new Adapters with a few lines of code. The foundation of this feature is Hana Smart Data Access. With this you can create virtual tables, which are views on top of remote source tables and read data from there.

For safety reasons these adapters do not run inside Hana but are hosted on one or many external computers running the Hana Data Provisioning Agent and the Adapters. This agent is a very small download from Service Market Place and can be located on any Windows/Linux computer. Since the agent talks to Hana via either TCP or https, the agent can even be installed inside the company network and loads into a Hana cloud instance!

Using that agent and its hosted adapters Hana can browse all available source tables, well in case of a RSS feed there is just a single table per RSS provider, and a virtual table being created based on that table structure.

Now that is a table just like any other, I can select from it using SQL, calculation views or whatever and will see the data as provided by the adapter. The user cannot see any difference to a native Hana table other than reading remote data will be slower than reading data from Hana.

That covers the batch case and the initial load.

For realtime Hana got extended to support a new SQL command "create remote subscription <name> using (<select from virtual table>) target <desired target>". As soon as such remote subscription got activated, the Adapter is asked to listen for changes in the source and send them as change rows to Hana for processing. The way RSS changes are received is by querying the URL frequently and push all found rows into to Hana. Other sources are might support streaming of data directly but that is up to the adapter developer. As seen from Hana the adapter provides change information in realtime, how the adapter does produce that we do not care.

This concludes a first overview about Hana Smart Data Integration. In subsequent posts I will talk about the use cases this opens up, details of each component and the internals.

Best Practice For Hana Performance Optimization (PART I):

Hi Everyone, I am sharing my experience of working on Oracle to Hana Migration Project.Below are the few points about the performance optimization of the sap hana code.

Points to consider when you are writing the sap hana procedure..!!

Please see the below mentioned steps.

1.      Always select only the required column instead of selecting all the columns.

Example: Suppose their are three table TABLE_A ,TABLE_B and TABLE_C with the below structure.

TABLE_A Structure:

TABLE_B Structure:

TABLE_C Structure:

Now suppose in your procedure you have to select only the Name,Salary and Designation from these three table based on the join condition and use them to populate the data into some target table TABLE_T.

So,For the given Scenario you should not use the below SQL Statements if you are using this it will lead to performance degrade of the particular procedure.

If you are using query like above then you are selecting more column than required. So its always better to select only the required columns which will result in performance improvement of your SQL procedures.

2.  Always try to use "NOT EXISTS" and "EXISTS" keyword in your procedure instead of  "NOT IN" and "IN" because using the      "NOT IN"  or "IN" inside the procedure will slow down the procedure performance.

     Example: I want to delete all the records from COMPONENT_A where ENTERPRISE ,SITE and PRODUCTION ORDER is not      in HEADER_A.

     Using the Below Delete statement will slow down the performance.

     So ,Its always advisable to use the NOT EXISTS statements like below which will improve the performance.

3.     Always try to avoid using HDBSEQUENCE in your procedure Becuase it will slow down your procedure performance.

     Example:- Suppose I have SALES table with below structure.

Now i want to select all the item from the sales table and add the suffix to all the item of sales table and scenario is one of the sales table column which value is constant.

Solution:-     So first solution which will come to our mind is to create a hdbsequence  and concatenate that sequence to Item column of the SALES table.

Steps are given as:

I.     Create a HDBSEQUENCE.

          a.     Go to Project and follow the steps to created the sequence as below.

II.     Now using the sequence created we can write the procedure for our scenario.Please see the below procedure using the sequence.

So, My observation was when i tried calling this procedure it took around 1 minute to execute. So i tried below approach.

If you have any column in your table which is constant through out you process then you should use row number function to achieve the same functionality. which will not affect the execution time at all. Like below.

So,When i executed the above procedure it took only few seconds.

So if anyone have better idea of removing the sequence from hana procedure,Please share you thoughts.

4.     Always try to avoid Combining the Join Engine and Calculation Engine of Hana. Though Hana execute calculation statements in calculation engine and join statements in join engine but its always better to separate the join engine and Calculation Engine Statements to provide better execution time.

Example:  In the below hana procedure I have used the table variable where we are storing the data from join of three table and their is calculation happening in the same join expression. Which means we are combing  join engine and calculation engine,Result more execution time.

CREATE PROCEDURE TEST_PROC

  LANGUAGE SQLSCRIPT

  SQL SECURITY INVOKER

  AS

BEGIN

JN_DATA  =      SELECT   T1.RUNTIME

                                         T2.ITEM,

                                         T3.LOCATION

                            FROM   DETAILS T1,

                                         ROUTING T2,

                                         RESOURCES T3

                             WHERE T1.BOR= T2.BOR

                             AND      T1.LOCATION = T2.LOCATION

                             AND      T1.SCENARIO= T3.SCENARIO

                             AND      T2.ITEM = T3.NAME

                             AND     T1.BOR LIKE '%BOR_ALT%'

                             AND     T2.BOS NOT LIKE '%_TMP_%'

                             AND     T3.ITEM = 'N' OR ITEM IS NULL;

INSERT INTO TABLE_COMPONENTS (SELECT * FROM :JN_DATA);

END;

Below Procedure where we have separated both the engine logic and it results in  faster execution of the procedure.

CREATE PROCEDURE TEST_PROC1

LANGUAGE SQLSCRIPT

SQL SECURITY INVOKER

AS

BEGIN

EXP_DETAIL    = SELECT RUNTIME,

                                         LOCATION,

                                         SCENARIO,

                                          BOR

                           FROM     DETAILS

                           WHERE  BOR LIKE '%BOR_ALT%';

EXP_ROUTING = SELECT   ITEM,

                                            LOCATION,

                                           BOR

                            FROM       ROUTING           

                            WHERE    BOS NOT LIKE '%_TMP_%';

EXP_RESOURCES= SELECT  NAME,

                                               RESOURCE,

                                              SCENARIO

                                              LOCATION

                                FROM         RESOURCES

                               WHERE     ITEM = 'N' OR ITEM IS NULL;

JOIN_DATA   = SELECT     T1.RUNTIME

                                          T2.ITEM,

                                          T3.LOCATION

                            FROM    :EXP_DETAIL T1,

                                         :EXP_ROUTING T2,

                                         :EXP_RESOURCES T3

                             WHERE  T1.BOR= T2.BOR

                             AND       T1.LOCATION = T2.LOCATION

                             AND       T1.SCENARIO= T3.SCENARIO

                             AND       T2.ITEM = T3.NAME;

INSERT INTO TABLE_COMPONENTS (SELECT * FROM :JOIN_DATA);

END;

So in the above procedure we have first calculated all the column then only we are using it for the join.which is why both the engine executes these statements separately ,result in better performance.

5.     Creating a read and write procedure is always better in terms of performance.So always try to create a read and write procedure to get the better performance.

       Example: Just for the example i am showing the procedure which takes more time when we use to read and write in the same procedure.

CREATE PROCEDURE HISTORY_DATA

  LANGUAGE SQLSCRIPT

  SQL SECURITY INVOKER

  AS

BEGIN

DATA_1=(SELECT  SCENARIOID,

                             LINENUM,

                            SITE,

                            NAME

                FROM HISTORY);

********************************************************

Many other Transaction on data not shown here

********************************************************

  INSERT INTO SHIPMENT_HISTORY

   (

    SCENARIOID,

    LINENUM,

    SITE,

    NAME

    )(SELECT * FROM :DATA_1);

  DATA_2=(SELECT      SCENARIOID,

                                    SHIPPED,

                                   DATESHIPPED,

                                  SOURCE,

                                  CREATEDDATE

                 FROM HISTORY);

********************************************************

Many other Transaction on data not shown here

********************************************************

INSERT INTO SHIPMENT_HISTORY

   (

    SCENARIOID,

    SHIPPED,

    DATESHIPPED,

    SOURCE,

    CREATEDDATE

    )(SELECT * FROM :DATA_2);

END;

So,the above procedure takes around 1:36 Minutes time when we run it that's the reason i have separated the procedure into read and write procedure.

READ PROCEDURE:  The read procedure in hana does not allow any DML statements inside the procedure.So we will just read the data from the target tables after all the transactions and pass that data to the output parameter of the procedure ,Output parameter of the procedure can be a scalar variable or table variable.

So below steps has to be followed to create the read and write procedure.

STEP I- First create the HDBTABLETYPE of the same column which you are passing to the output parameter. And to Create the HDBTABLE first we have to declare the artifacts of different datatypes which we can use to create the table type.As shown in the below screen shot.

STEP II- Now create the table type using these artefacts like below.      

STEP III- Create a read procedure and pass the data to output variable of above table type.

CREATE PROCEDURE HISTORY_DATA_READ

                                                                           (OUT OUT_DATA_1FULL_PATH_OF_HDBTYPE_HISTORY_1,

                                                                            OUT OUT_DATA_2 FULL_PATH_OF_HDBTYPE_HISTORY_2)

  LANGUAGE SQLSCRIPT

  READS SQL DATA

  SQL SECURITY INVOKER

  AS

BEGIN

********************************************************

Many other Transaction on data not shown here

********************************************************

--final data to be sent to out parameter

DATA_1=(SELECT  SCENARIOID,

                              LINENUM,

                              SITE,

                              NAME

                FROM HISTORY);

********************************************************

Many other Transaction on data not shown here

********************************************************

--final data to be sent to out parameter

  DATA_2=(SELECT    SCENARIOID,

                                   SHIPPED,

                                   DATESHIPPED,

                                  SOURCE,

                                  CREATEDDATE

                 FROM HISTORY);

END;

WRITE PROCEDURE:- Now read procedure is created so we will create one procedure which will call the read procedure and we will read the data into another variables which we will use to insert into target tables.

CREATE PROCEDURE HISTORY_DATA

  LANGUAGE SQLSCRIPT

  SQL SECURITY INVOKER

  AS

BEGIN

--call the read procedure to store the data into two table variables

***************************************************************************

CALL HISTORY_DATA_READ (DATA_1_IN,DATA_2_IN);

***************************************************************************

  INSERT INTO SHIPMENT_HISTORY

   (

    SCENARIOID,

    LINENUM,

    SITE,

    NAME

    )(SELECT * FROM :DATA_1_IN);

INSERT INTO SHIPMENT_HISTORY

   (

    SCENARIOID,

    SHIPPED,

    DATESHIPPED,

    SOURCE,

    CREATEDDATE

    )(SELECT * FROM :DATA_2_IN);

END;

So now after separating the procedures into read and write it took only 2.01 Seconds to execute.Conclusion is its always better to use read and write procedure.

So,these are the some points from my work experience on Oracle to Hana Migration Project.

Please share your thought about the post,Advise for further improvement is most welcome ..:)

I will release the Part II very soon..:)

Happy Reading..:)

Over the last few years the reputation of Hana constantly grew and other people found various arguments why Hana is nothing better. Me being an Oracle expert for more than 20 years was among them, I have to admit. Looking back it was rather a lack of understanding on my side and being trapped in marketing statements of SAP. You think that is a lame excuse? You are probably right.

So let me take you on my journey with Hana and share some internals you have never read elsewhere before.

The first argument came in 2010, mostly from Oracle, and back then was - understandably - "Of course keeping everything in your head is much faster, but that is simply not doable." If I may translate that, the statement was: memory is too limited and too expensive. True enough, even today. What is the price of a hard disk with 1TB and, in comparison, how much does a server with 1TB of memory cost? A completely reasonable argument.

Actually, I just digged up a youtube video and it is fun to watch, even today.

SAP was arguing at the time that you compress the data and hence you do not need as much memory. We all know how compression works and the costs involved, I found that not very convincing.

What struck me even more however was the fact that traditional databases do cache the data in RAM as well, so they are in-memory so to speak, except that only the frequently accessed data is cached, the archived data does not eat into your memory budget.

What I hated about Hana the most was the row versus column storage. Marketing touted that thanks to the columnar storage you can aggregate data much faster, and when confronted with the question of reading an entire row with all columns, the response was "we have row storage as well". Excellent answer. Now I would need both, a  row and columnar storage for each table? You cannot be serious.

With this kind of mindset I started to look at Hana, did typical performance tests and quickly found out, there is something severely wrong with my assessments of Hana. Thankfully the Hana developers took the time to engage with me and provided me with internals that explained what I missed before.

Let me try to show.....

The three technologies

According to marketing the benefit of Hana are the top three technologies shown below. I am sure you had been bombed with the same arguments, Hana is In-Memory and therefore it is fast. And Columnar. It does Compression!

I can fully understand now, why people, including myself, were skeptical. All of these technologies are nothing new as Larry Ellison stated in above video mentioning Oracle's TimesTen product as an example.

The secret to the Hana performance is not the three technologies as such, in my opinion, it is the intelligent combination of these three plus the insert-only approach. The reason I am saying that is when looking at each technology individually, all have advantages but sever disadvantages as well. Memory is limited, Compression is CPU intensive. Columnar Storage puts column values closely together whereas row storage provides each row's data as one block. Insert-only requires to drop outdated versions from time to time.

In-Memory
Compression
Columnar Storage
Insert-Only

In-Memory

The basic idea is that memory is much faster than disk. Actually it is times faster. A 2014 CPU has a memory bandwidth of 10GByte/sec and higher, a single disk around 150MByte/sec - difference by factor 70. If your program is using the same memory frequently, it is cached inside the CPU's L1 or L2 cache, speeding up the memory bandwidth by another factor of 10. On the contrary the disk speed of 150MB/sec is for sequential read access only, random access is times worse for a disk system whereas has no negative impact on the RAM.
The downside of memory is the costs of the memory chip itself (7USD/GByte for RAM compared to 0.05USD/GByte for disks as of 2014) and the hardware platform you need, in order to cope with more memory, is getting increasingly more expensive also.
On the other hand, if I need 1TB of RAM that would be 7000USD. While this is much money compared to a single 100USD disk, it is not much in terms of absolute numbers.
But you can turn around my argument and simply say, if you have a 1TB big database one disk, use a server with 1TB of memory so all data can be cached.

So the argument "in-memory!" cannot be the entire truth.

Compression

The idea of compression is simple, a single CPU is much faster than the memory bus and the disk, not to mention that multiple CPUs share the same bus, hence compressing data in order to reduce the amount of data is beneficial as long the overhead of that is not too huge. Therefore every major database supports compression. It is not very popular though as compressing a database block and decompressing it takes its toll. The most obvious cost overhead is when data is updated inside a database block. You have to uncompress the database block, make the change and then compress it again. Hopefully it fits into the same database block still otherwise you need a second one.

So the argument "Compression!" cannot be the entire truth.

Columnar storage

For a simple select sum(revenue) the columnar storage is just perfect. You have to read one column only, hence just a fraction of the whole table data is needed. Imagine you have all the data of one column in one file, this will be much faster as with traditional row orientated tables, where all the table data is in one file (or database object to be more precise) and you have to read the entire table in order to figure out each row's columns value.
In case you want to see all columns of a single row like it is typical for OLTP queries, the row storage is much better suited.

So the argument "Columnar Storage!" cannot be the entire truth.

Insert only

A real database should have read consistency in the sense of when I execute my select statement I get all committed data of the table but neither will I see data that has been committed after nor will my long running query fail just because the old value was overwritten by the database writer.
The only major database I know supporting that since the beginning is Oracle (SQLServer has a option to turn that on/off), but you have to pay a price for that consistency. Whenever the database writer does overwrite a block with new data, the old version has to be kept somewhere, in the rollback segments of the database in case of Oracle. So a simple update or insert into an existing block requires two operations, the actual change plus saving the old version.
With insert only, the idea is a different one, there in each table the old data is never overwritten but only appended. If you execute an update against an existing row, a new row is written at the end of the table with same primary key value plus a transaction number. When a select statement is executed, it will see multiple versions and use the one with the highest transaction number less or equal to the currently active global transaction number in the database.
There is a tradeoff hence, the tables grow fast with this approach, especially when you update just a single byte, but on the other hand you are faster.

So the argument "Insert only!" cannot be the entire truth.

Combining the technologies

Okay, this was my starting point. All technologies are good ideas, other companies tried these as well and have built proper niche products, mostly around the analytic use cases. One database is a pure in-memory database that needs a regular database to persist the data, others you can insert data but not update or delete. Many support compression but usually it is turned off by customers.

The claim of Hana is to be a database that supports analytic and transactional use cases and is better than other databases in all areas. That should be easy to be put into perspective, I thought.

However, the one thing I did overlook at that time was how these technologies benefit from each other. So let us go through a couple of mutual benefits to get the idea.

Combination of Compression with Columnar Storage +

Compressing data you can do best whenever there is a repetitive pattern. Let us have a look at a real example, the material master table.

What can you compress better, the entire file or a single column?
The answer is obvious but nevertheless I exported these columns from the MARA table, all 20'000 rows of my system, into a CSV file (1'033KB big) and did zip the one file with all data plus nine files with one column each.

Obviously the primary key cannot be compressed much, it is half of the data in fact but all other columns are, the MAND file is 303 byte large, the ERSDA file with all the many create dates is 12'803 bytes big.

But this is not a fair comparison as the zip algorithm favors larger datasets as it can look for patterns easier and it is a fairly aggressive algorithm. In databases your compression is lower to require less CPU cycles and more important, each file is split into database blocks. Meaning that if you have a fairly wide table, one database block might include one row of data only that is compressed - hence almost no compression possible at all.
With columnar storage we have no issue with that side effect.

So as you see, the technology of using columnar storage has a huge positive side effect on the degree compression is possible.

Combination of Compression with In-Memory +

That's an easy one. The more compression we do the less memory we need. Before we said that the cost ratio between disk and RAM is about 700 times cheaper. Thanks to the compression, usually is a factor of 10, the disk is just 70 times cheaper. Or more important, for your 10 TB database you do not need a server with 10TB of RAM which would be very expensive.

Combination of Compression with Insert-only +

Compression has one important downside as well however, what if a row is updated or deleted even? The compression spans multiple rows, so when you change a value you have to uncompress the entire thing, change the value and compress it again. With traditional databases you do exactly that. Okay, you do not uncompress the entire table, the table data is split into pages (database blocks) and hence only the impacted page has to be recompressed but still you can virtually watch how much these indexes slow down updates/deletes in traditional databases.
So what does Hana do? Quite simple, it does not update and delete the existing data. It appends the change as a new version with a transaction ID and when you query the table, you will read the oldest version of each row, the oldest version that matches the query execution start time. So suddenly no recompression is needed anymore, data is appended uncompressed to the end of the table and once the uncompressed area exceeds a limit, it gets compressed and new data is inserted into a new page of the table.
The other advantage of that approach is, if a single row is updated multiple times, which row will that be? A booking made 10 years ago? Very unlikely. It will be a recent one, one that is still in the uncompressed area likely.

Combination of Columnar Storage with Insert-only +

As the data is inserted at the end only, finding the data of a row in the various columns is very simple. According to the current transaction id, the row to read was the version at the table position 1'234'567, hence the task is to find for each column the value at that position.

Just imagine an Excel sheet. What is faster: Reading the cells (A,10) + (B,10) + (C,10), in other words the row 10 with the three columns A, B and C?

Or reading the the cells (J,1) + (J,2) + (J,3), in other words the column J with the values in the three rows 1,2 and 3?

It does not make a difference. None at all. The entire argument of reading row-wise is better is actually based on the assumption that it is faster to read horizontally than vertically. Which is true, if the data is stored on disk in a horizontal fashion. Then the data of one row is closely together and hence read in one rush from the disk cylinder. But on a memory system it does not matter at all.

Putting all four together +++

What was the issue with compression?

• Compression works best on similar data -> one column often has similar values -> solved
• Recompression in case something does change -> we do no change data but insert only -> solved
• Compression is CPU expensive -> not a pkzip like compression is used but dictionary and pattern compression -> faster than reading the plain data

What was the issue with memory?

• More expensive that disk -> thanks to the compression that factor is dampened -> Reality had proven Hana can run even large enterprise ERP systems

What was the issue with Columnar Storage?

• You need to locate the row value for each column individually -> But actually, for memory it does not matter if you read two words from nearby or far away memory pointers. With compression this might even be faster!
• Changing values requires to change the entire column string -> True, hence Hana does not change values, it appends data only.

What is the issue with Insert-only?

• More and more old data is present and needs to be removed or memory consumption grows fast -> The most recent data is not added into the compressed storage, it is kept in the delta storage. All changes within that delta storage are handled to allow updates.
• Above problem is faced only if changes are made on rows in the compressed storage area -> less likely but possible.

Comparing Hana with other databases

Meanwhile other database vendors have been faced with the realities as well and we have seen various announcements to jump on the in-memory bandwagon as well. Given your knowledge about the Hana internals now, you should be able to quantify the advantage for yourself.

Oracle: With Oracle 12c there is an in-memory option available. This option allows you to store the data in addition to the traditional disk based way in a in-memory area as well. Personally I have mixed feelings about that. On the one hand it is very convenient for existing Oracle users. You execute a couple of statements and suddenly you are times faster with your queries.

But this assumes that the traditional disk based storage does have advantages and I tried to show above it does not. Hence it is a workaround only, kind of what I criticized when the response was "Hana has column and row storage". And it leaves room for questions like

• What are the costs to store the data twice when inserting/updating data?
• Do what degree is the database slowed down if part of the memory used for database block caching is now reserved for in-memory storage?
• No question this will be faster on the query side, but the OLTP load worries me.

Do me that sounds like the dinosaurs way, trying to sound flexible when all you do is actually doubling the costs. Obviously I lack the hands on experience but sorry Oracle, I was a big fan of yours but that does not sound like a master minds plan to me.

Nice reading I found: Rittman Mead Consulting: Taking a Look at the Oracle Database 12c In-Memory Option

Do you concur with my opinion?

Big Data is about VOLUME, VARIETY and VELOCITY of data. Let us see how SAP HANA platform full fill the requirement of 3 V’s (Volume, Variety and velocity) challenges of Big Data.

VOLUME

Volume of data increasing day by day and by 2020 it will be 40 Zetabyte .So for Big data now challenge is to store high volume of data.SAP HANA has successfully overcome with the volume aspect of Big Data by fortifying  SAP HANA platform. Following are the two game changing features in SAP HANA platform related to data volume.

• SAP HANA and HADOOP integration
• Dynamic Tiering

SAP HANA and HADOOP integration

            HADOOP facilitate to store infinite volume of data using distributed file system.SAP with its release of SP09, very tightly integrated with hadoop.Following are the SAP HANA and HADOOP integration options:

• SDA (Smart Data Access)
• SAP Data Services
• SAP BO-IDT (Information Design Tool)
  
• HANA XS Engine and Hadoop Hbase

SMART DATA ACCESS:Smart Data Access (SDA) provides SAP HANA with data virtualization capabilities. This technology allows to create a virtual table to combine SAP HANA data with other heterogeneous data sources like-HADOOP,TERADATA,MS SQL SERVER,ORACLE,SAP Sybase ASE,SAP Sybase IQ,SAP HANA

          In SAP HANA SPS07, HANA connect to HIVE:

• Hadoop Pig and Hbase does not support ODBC but HIVE does  support ODBC. SAP HANA can connect to Hadoop HIVE virtually using SDA. Please refer installation and configuration guide of Hadoop and SAP HANA using SDA.
  • SAP HANA Smart Data Access using Hadoop Hive.pdf
• SAP HANA Acedmy video tutorial to configure Hadoop HIVE with SAP HANA using SDA.
  • SAP HANA Academy - SDA: Hadoop installation (1/4) - YouTube
    
  • SAP HANA Academy - SDA: Loading Data into Hadoop (2/4) - YouTube
    
  • SAP HANA Academy - SDA: Configuring ODBC drivers (3/4) - YouTube
  • SAP HANA Academy - SDA: Using the remote data source (4/4) - YouTube                                  
    
• SQL Script to create Remote Data Source to HADOOP HIVE

CREATE REMOTE SOURCE HIVE

ADAPTER "hiveodbc"

CONFIGURATION 'DNS=HIVE'

WITH CREDENTIAL TYPE 'PASSWORD'

USING 'user=hive;password=hive';

• Create Virtual table on HIVE remote data source and consumed it on HANA catalog.

            In SAP HANA SPS08, HANA connect to Apache SPARK:

• SQL Script to create Remote Data Source to HADOOP SPARK

CREATE REMOTE SOURCE HIVE

ADAPTER "hiveodbc"

CONFIGURATION 'DNS=SPARK'

WITH CREDENTIAL TYPE 'PASSWORD'

USING 'user=hive;password=SHa12345';

• Create Virtual table on SPARK remote data source and consumed it on HANA catalog.

• SAP HANA Acedmy video tutorial to configure Hadoop SPARK with SAP HANA .
  • HANA Academy: SAP Smart Data Access for Apache Spark - Video 1 of 4. - YouTube
  • HANA Academy: SAP Smart Data Access for Apache Spark - Video 2 of 4. - YouTube
  • HANA Academy: SAP Smart Data Access for Apache Spark - Video 3 of 4. - YouTube
  • HANA Academy: SAP Smart Data Access for Apache Spark - Video 4 of 4. - YouTube

            In SAP HANA SPS09, HANA directly connect to Hadoop HDFS:

• Create Map Reduce Archives package in SAP HANA Development Prospective using JAVA
• Create Remote Data Source directly to Hadoop HDFS

CREATE REMOTE SOURCE HADOOP_SOURCE

ADAPTER "hadoop"

CONFIGURATION 'webhdfs_url=<url:port>;webhcat_url=<url:port>'

WITH CREDENTIAL TYPE 'PASSWORD'

USING 'user=hive;password=hive';

• Create Virtual Function

CREATE VIRTUAL FUNCTION HADOOP_WORD_COUNT

RETURN TABLE ("word" NVARCHAR(60),"count" integer)

package DEV01."DEV01.HanaShared::WordCount"

CONFIGURATION 'enable_remote_cache;mapred_jobchain=[("mapred_input":"/data/mockingbird"."mapred_mapper":"com.sap.hana.hadoop.samples.Wordmapper",

"mapred_reducer":"com.sap.hana.hadoop.samples.WordReducer"}]'

AT HADOOP_SOURCE;

• Create Virtual UDF to directly connect to HDFS file.

CREATE VIRTUAL FUNCTION HADOOP_PRODUCT_UDF()

RETURN TABLE ("product_class_is" INTEGER, "product_id" INTEGER,"brabd_name" VARCHAR(255))

CONFIGURATION 'datetiem_format=yyyy-MM-dd HH:mm:ss;date_format=yyyy-mm-dd HH:mm:ss;time_format=HH:mm:ss;enable_remote_caching=true;cache_validity=3600;

hdfs_location=/apps/hive/warehouse/dflo.db/product'

AT HADOOP_SOURCE;

• SAP HANA Acedmy video tutorial to configure Hadoop HDFS with SAP HANA .
  • SAP HANA Academy: SDA; Hadoop Enhancements - 1. Creating User [SPS09] - YouTube
  • SAP HANA Academy: SDA; Hadoop Enhancements - 2. Map Reduce Archives [SPS09] - YouTube
  • SAP HANA Academy: SDA; Hadoop Enhancements - 3. Remote Sources [SPS09] - YouTube
  • SAP HANA Academy: SDA; Hadoop Enhancements - 4. Virtual Functions [SPS09] - YouTube
  • SAP HANA Academy: SDA; Hadoop Enhancements - 5. Virtual UDFs [SPS09] - YouTube

CONNECT TO HADOOP USING SAP DATA SERVICE         

• Select File Format tab from Local Object Library->right click on HDFS File and click New

• Provide following parameter values in HDFS File Format editor
  • Name: HDFS
  • Namenode host: <host name of hadoop installation>
  • Namenode port: <hadoop port>
  • Root Directory: < Hadoop file path>=</user/hadoop/input>
  • File Name: hdfs_data.txt

• Click on Save&Close and double click on created HDFS file again to view file format.

• Cretate Project->Job->Data Flow
• Drag HDFS file to the canvase and make it as source->drag query transformation and target table on the data flow canvase and join.

• Double click on Query transformation and schema IN and schema out

• Execute Job and view the data brought in HANA from Hadoop.

• Please refer HANA Academy Video
  • SAP HANA Academy - Using Data Services to import data from a HADOOP system - YouTube

SAP BO(IDT)-HADOOP INTEGRATION

• We can create universe on Hadoop HIVE using SAP BO IDT(Information Design Tool).Please read complete doc for installation and configuration of SAP BO IDT on HIVE.
  • Creating_a_universe_on_Hadoop_Hive_Jan_2014.pdf

HANA XSENGINE AND HADOOP HBASE

                    HANA XSEngine can talk to Hadoop Hbase via server side Javascript.Please refer following article for more details.

                    Streaming Real-time Data to HADOOP and HANA

DYNAMIC TIERING

          Dynamic tiering is SAP HANA extended storage of SAP IQ ES server integrated with SAP HANA node.Dynamic tie-ring has been included in SPS09.HOT data reside in SAP HANA In-Memory and warm data reside on IQ ES server columnar petabyte storage on disk.It provides environment to increase Terabyte SAP HANA In-Memory capability to Patabyte columnar disk storage without using Hadoop.

     HOT & WARM Table creation:

CREATE TABLE "SYSTEM".SalesOrder_HOT" (

"ID" INTEGER NOT NULL,

"CUSTOMERID" INTEGER NOT NULL,

"ORDERDATE" DATE NOT NULL,

"FINANCIALCODE CHAR(2) NULL,

"REGION" CHAR(2) NULL,

"SALESREPRESENTATIVE" INTEGER NOT NULL,

PRIMARY KEY("ID")

);

CREATE TABLE "SYSTEM".SalesOrder_WARM" (

"ID" INTEGER NOT NULL,

"CUSTOMERID" INTEGER NOT NULL,

"ORDERDATE" DATE NOT NULL,

"FINANCIALCODE CHAR(2) NULL,

"REGION" CHAR(2) NULL,

"SALESREPRESENTATIVE" INTEGER NOT NULL,

PRIMARY KEY("ID")

)USING EXTENDED STORAGE;

Reference Document:

SAP HANA SPS 09 - Dynamic Tiering.pdf

Reference SAP HANA Academy Video:

SAP HANA Academy - SAP HANA Dynamic Tiering : Installation Overview [SPS 09] - YouTube

SAP HANA Academy - SAP HANA Dynamic Tiering: Introduction [SPS09] - YouTube

The EPM sample data included with the SHINE demo application is a helpful resource for working on development and modeling scenarios.

Included in the sample data are supporting tables that are required to perform Unit of Measure and Currency conversions in analytic and calculation views, as stated in the SAP HANA Modeling Guide.

However, the sample data is missing table T006D.  Without this table, an error is generated when attempting a Unit of Measure conversion.

The attached file has the necessary SQL DDL and DML scripts to create and load a column table (and create a related synonym) for T006D.  The scripts can be executed from a SQL Console view in the SAP HANA Modeler perspective.

After refreshing the schema, T006D is displayed and can be referenced for Unit of Measure conversions.

(Note: T006D table structure and default data obtained from SAP BW 7.4 SP8.)

The success of Linux adoption within SAP

Background

2 years ago, SAP Global IT Infrastructure Service – Service Center Labs IT, took the challenge of simplifying Linux OS deployments (OSD) in the area of developer workspace environment.

Until then, there was neither Linux OSD service nor Linux support provided by SAP Global IT
in this area. This means that each developer who needed access to Linux OS, spent valuable time
installing his own Linux system.

From IT management perspective, there was no control over these systems – it was not
secure, not conforms to any guidelines, not managed or inventoried.

Together with Senior Linux consultant Shay Cohen of G.S.R. IT consulting Ltd., Labs IT designed and built a flexible and scaleable service to
manage and deploy Linux systems in automated way.

When we designed the service, our goal was to provide the end user with a system which is preconfigured and ready for use out of the box. We focused on two main aspects:

1. Conformity with SAP Global IT standards (e.g. systems naming conventions, security requirements, system settings)
2. Simplicity:
   1. For IT to deploy
   2. For end user to request and use

How we achieved this?

Using native Linux tools, Linux Deployment Tool Kit was built and supported the following process:

The first step of the process after the end user submitted service requires, is the key for the auto configuration and for the out of the box experience we wanted to achieve. In this step, IT technician enters LDT deployment task. In order to enter it, the following data should be provided:

1. User ID from Active Directory which will be defined with elevated permissions on the system.
2. MAC address of the target system for Linux OSD.
3. Location of the target system
4. Equipment No.(Internal asset number) of the target system. This will be used to configure the hostname according to SAP IT naming convention.
5. System type – VM, Desktop or server – this will affect the way the system will be configured.E.g. different hostname, VMWare tools installed/not installed etc.
6. SWAP File size.
7. Required Linux distribution (SUSE/Redhat etc.)
8. Profile – preconfigure set of tools which will be installed on the system.

With this information in the DB the system can be fully automatically installed and configured – ready for use out of the box!

This process enables us to reach the goals we set:

1. Conformity with SAP Global IT standards:
   1. Each Linux system which is deployed via LDT is
      automatically configure – hostname, DNS settings, input local etc. are
      configured according to the deployment task which is entered via SAP IT OS
      deployment portal.
   2. McAfee
      Anti-Virus agent is installed and managed centrally by SAP Global IT Client
      Protection team.
   3. LDT Agent is installed. This agent is the system
      management interface for Labs IT. It checks periodically for tasks waiting for
      the systems and reports back to LDT DB about the system information, heartbeat,
      Anti-Virus agent status and tasks execution results.
   4. Scrambled root
      password with a local rescue account with periodically changing password to
      enable IT support login.
   5. Integration with
      SAP Active Directory domain.
2. Simplicity:
   1. For IT to deploy – all is required from IT support technician who
      deploys Linux is to enter the required information in SAP IT OSD Portal and
      create a LDT deployment task. Afterwards, the OSD process run automatically
      after the technician boots the system with LDT boot ISO.
   2. For end user to request and use – all it takes for the end user to
      request Linux system is to enter an IT Service request with his used ID and the
      equipment number of his system. Afterwards, he is shipped with a system
      which is ready for use out of the box– just login with your domain account
      and password and start working!

Adoption of the service

The service was very successfully adopted by IT teams as well as our customers – SAP HANA developers any other development/QA teams who needs to work with Linux.

Since the service went live in October 2012 over 1,400 LDT OSD took place. Below the monthly deployment trend is presented for the last 5 months of 2013. The screen shot is captured from LDT inventory portal:

In LDT portal, we can also track the number of live systems. These are system which reported back to the system in the last 24Hrs. this dashboard present the number of live systems, deviation by geographical region, distribution and type:

Summary

As SAP HANA took place in SAP strategy, the demand from HAVA developers for Linux systems increased drastically, and especially for SUSE Linux.

With LDT service in place, SAP Global IT was ready to support this growing demand with simple to use service.

HANA developers have access Linux systems at the tip of their fingertips, reducing the time it takes them to
setup these systems from few hours to few minutes.

Semantic Code Completion in SAP HANA Studio

The fact that we have such long names for tables, views and table types in HANA, has been a pain point for many for some time now.  To help alleviate this issue, we have built semantic code completion into the SQLScript editor in SAP HANA Studio.  Now when a developer needs to do a SELECT against a particular table, he can hit CTRL+SPACE and get a list of tables to choose from.   The list is compiled of relevant objects based on the context of the statement, so if you have a SELECT statement and have entered the name of the schema already, and hit CTRL+SPACE, you will only get a listing of tables from that particular schema. This also works when calling procedures, or defining parameters with global table types.

Check out the demo video here.

SQLScript Editor & Debugger in the SAP Web-Development Workbench

Prior to SPS9, there was no way to maintain procedures from the SAP Web-based Development Workbench. You were forced to use the SAP HANA Studio for this.  Now as of SPS9, we have a basic SQLScript editor for maintaining .hdbprocedure files.  The .procedure file format is not supported here.  This editor has basic keyword code hints and syntax highlighting.

Since we can now create procedures from the SAP HANA Web-Based Development Workbench, it makes sense that we should be able to debug them.  As of SPS9, we also have a SQLScript procedure debugger as well. Currently, you must set breakpoints in the runtime object in the catalog, and then CALL your procedure from the SQLConsole in order to debug it.  We have plans to make it possible to debug a design time artifact directly without having to drop to the runtime object.  Within the debugger, you can of course single step through the code, and evaluate input/output parameters as well as intermediate scalar and table variables.

See a demo video here.

Table Type Definitions for Parameters

In previous support packages, we’ve had several different ways to create and reference table types when defining input/output parameters in our procedures.   For .procedure file format, we had “local table types” which really were not local, which is why we did not support them in the new .hdbprocedure file format.  For .hdbprocedure files, we recommended to create your tables types globally via CDS(.hdbdd file).  While I will still recommend to create table types via CDS for global type scenarios,  I am pleased to announce that we now have the possibility to declare local table types inline for parameters.  In the screen shot below you will see that I have an OUT parameter called EX_PRODUCT_SALE_PRICE which has a table type definition using the keyword TABLE followed by the column list with associated simple types.  These type declarations are truly local and cannot be used across procedures.  For situations where that you know that your table type will not be reused frequently, it might make sense and be a little easier to simply define the structure inline as opposed to creating it via CDS.

Table Type Definitions for Variables

In previous support packages, intermediate table variables were simply defined by the result set of the data selection, such as a SELECT statement.  So whatever columns were in the field list would become the structure of the intermediate table variable.  The issue with this approach is that there is some performance cost associated with the type conversions at runtime.  Also, this could cause some ambiguousness in the code.  As of SPS9, we can now explicitly define the structure of an intermediate table variable from the DECLARE statement.  As shown below, I have an intermediate table variable called LT_PRODUCTS which is defined as a TABLE, followed by the column list and associated simple types.  This allows the developer to have strict typing within the procedure and avoid any unnecessary performance costs from type conversions.

Autonomous Transactions

Another new language features in SPS9, is Autonomous Transactions.  The autonomous transaction allows the developer to create an isolated block of code which runs as an independent transaction.  This feature is particular helpful when executing logging type tasks.  Committed statements inside the autonomous transaction block will be persisted regardless of a rollback of the main transaction.  The keywords COMMIT and ROLLBACK are only allowed within the autonomous transaction block and not in the main line of the procedure.  If any tables are updated within the main body of the procedure, those tables are not allowed to be accessed from within the autonomous transaction block.

See a demo video of the new language features here.  And for more information regarding the SQLScript language itself, please check out the SQLScript Reference Guide.

Use of CE Functions within Procedures & Scripted Calculation Views

Although not specific to SPS9, I’d like close with some clarification around the use of CE Functions. Calculation Engine(CE) Functions, also known as Plan Operators, are an alternative to writing SQL.  At one time, it was recommended to always use CE Functions over SQL in both SQLScript stored procedures as well as scripted calculation views as they performed better than SQL. This is no longer the case.  The recommendation moving forward is to use SQL rather than CE Functions within SQLScript. The execution of Calculation Engine Functions currently is bound to processing within the calculation engine and does not allow a possibility to use alternative execution engines, such as L native execution. As most Calculation Engine Functions are converted internally and treated as SQL operations, the conversion requires multiple layers of optimizations. This can be avoided by direct SQL use. Depending on your system configuration and the version you use, mixing Calculation Engine Functions/Plan Operators and SQL can lead to significant performance penalties when compared to plain SQL implementation. Please note that the above described recommendation/behavior only applies to calculation engine functionality exposed by SQLScript.  Therefore only SQLScript related artifacts such as procedures, table functions and scripted calculation views are affected.

In this blog, I will collect the various smaller blogs that detail all the new developer related features in SAP HANA SPS 09.  This will be a "living" document which is updated as new blogs are released.

HANA Programming Model

Miscellaneous Improvements

The following are various items that don't really have a category of their own.

• New Mozilla VM (currently version 28)
• Relaxed strict mode settings of the JavaScript VM
• New threading model implementation internally which lays the foundation for future features. No changes to the programming model itself due to these changes yet in SPS 09.  Some general performance improvements thanks to this change.

Miscellaneous Security Features

In this blog we will have a first look at the new miscellaneous security features added to development model in SAP HANA SPS 09.

SAP HANA SPS 09: New Developer Features; Miscellaneous Security Features

New XSJS Database Interface

In this blog we will have a first look at the new XSJS database interface in SAP HANA SPS09.  This is a completely redesigned and rebuilt database interface which replaces the current implementation in the $.db package.  This new interface, which is available as a separate API in $.hdb, focuses on several key areas of improvements.

SAP HANA SPS 09: New Developer Features; New XSJS Database Interface

New Core XSJS APIs

Already we have looked at the new XSJS Database Interface in SPS 09. However this isn't the only new core XSJS API in SPS 09.  There are several other new core XSJS APIs which we will now explore further in this blog.
SAP HANA SPS 09: New Developer Features; New Core XSJS APIs

New XSODATA Features

In this blog we will look at new features in the XSODATA service framework in SAP HANA SPS 09.

SAP HANA SPS 09: New Developer Features; New XSODATA Features

SQLScript

New features in SQLScript language and tools:
New SQLScript Features in SAP HANA 1.0 SPS9

XS Admin Tools

Coming Soon

HANA Test Tools

Coming Soon

Core Data Services

Coming Soon

XSDS (XS Data Services)

Coming Soon

Repository REST API

Coming Soon

SAP River

Coming Soon

HANA Development Tools

SAP HANA Web-based Development Workbench

With SPS 09 we continue to enhance the browser based development tools adding support for a larger number of development artifacts as well as enhancing and improving the editors which already existed in previous releases.
SAP HANA SPS 09: New Developer Features; SAP HANA Web-based Development Workbench

SAP HANA Studio

While we see major investment in the web-based tooling around SAP HANA, SAP also continues to make improvements and additions to the Eclipse based SAP HANA Studio as well. In this blog we will detail the enhancements to the SAP HANA Studio.

SAP HANA SPS 09: New Developer Features; SAP HANA Studio

This blog is part of the larger series on all new developer features in SAP HANA SPS 09: http://scn.sap.com/community/developer-center/hana/blog/2014/12/02/sap-hana-sps-09-new-developer-features

In this blog we will have a first look at the new miscellaneous security features added to development model in SAP HANA SPS 09.

Full CORS (Cross-Origin Resource Sharing) Support.

Since SPS 06, we've had basic CORS support which could be configured at the package level.  This support allowed you to either enable or disable CORS, but in SPS 09 we expand the configuration options to allow filtering by origins, headers and http methods.

Custom Headers/X-Frame

This new feature allows you to control if the browser should allow a page within this HANA page to be rendered within a frame, iframe, or object.  This helps to avoid clickjacking attacks by keeping content from being embedded within a malicious site.

Possible values:

• DENY The page cannot be displayed in a frame, regardless of the site attempting to do so.
• SAMEORIGIN The page can only be displayed in a frame on the same origin as the page itself.
• ALLOW-FROM uriThe page can only be displayed in a frame on the specified origin. In other words, if you specify DENY, not only will attempts to load the page in a frame fail when loaded from other sites, attempts to do so will fail when loaded from the same site. On the other hand, if you specify SAMEORIGIN, you can still use the page in a frame as long as the site including it in a frame is the same as the one serving the page.

Various Authentication Features

• Secure HTTP Session Cookies
• Support for Clietn Certificates from F5's Big IP
• SAML Single Logout (SLO) support
• SAML Authentication in Authorization header

Support for Virus Scan Interface (VSI) for applications

New XSJS API ($.security.AntiVirus) to access and use the SAP Virus Scan Interface from your server side JavaScript coding.

• The scan needs a Virus Scan Adapter (VSA) to be installed on the host
• The setup and configuration is available with SAP note 2081108
• This class uses the SAP certified interface NW-VSI 2.00 (see SAP note 1883424)
• For a list of the AV products supported, see SAP note 1494278

Code Sample for using the new Virus Scan Interface from XSJS:

try {  //create a new $.security.AntiVirus object using the default profile  var av = new $.security.AntiVirus();  av.scan($.request.body);
} catch (e) {  $.response.setBody(e.toString());
}