The GigaSpaces Blog

GigaSpaces’ upcoming cloud framework

Dekel Tankel — Sat, 06 Sep 2008 02:43:00 +0000

Recently I had the pleasure of doing a demo to one of our partners using the new “cloud framework”.This framework brings forth 3 key concepts, which we think are essential to successfully develop, test and deploy any scalable transaction applicatio...

GigaSpaces & Maven Setup : a mini-recipe

Owen Taylor — Thu, 28 Aug 2008 14:28:47 +0000

When trying out the bundled Maven tool that now ships with GigaSpaces, I found it friendly to my sensibilities to create a couple of new scripts:

I placed both of these in the GS_HOME/tools/maven directory:

The first one sets up the necessary paths before calling the installmavenrep.bat that comes with GigaSpaces:

mavensetup.cmd



	set M2_HOME=%~dp0apache-maven-2.0.9

set PATH=%PATH%;%M2_HOME%\bin

call installmavenrep.bat

The second one starts up a new shell with the paths set so that maven commands will function as expected by the GigaSpaces documentation:

mavenshell.cmd



	set M2_HOME=%~dp0apache-maven-2.0.9

set GS_HOME=%~dp0..\..

set PATH=%PATH%;%M2_HOME%\bin;%GS_HOME%\bin

	start "MAVEN SHELL"

Once in my “MAVEN SHELL”, I can create a new application that uses the mirror service by executing:
C:\gigaspaces-xap-6.6.0-m3\tools\maven>mvn os:create dashDgroupId=com.test.mirror dash
DartifactId=MyFirstMirrorApp dashDtemplate=basic-async-persistency

With the project created – and quickly too! I now add another script to the mix:

This script lives in the root directory of the newly generated project.

In this example that would be the MyFirstMirrorApp directory found on my machine here:

C:\gigaspaces-xap-6.6.0-m3\tools\maven\MyFirstMirrorApp>

The purpose of this script is to start up each of the aspects of the project in the proper order and demonstrate that it all works:

(Note: I execute this from within my “MAVEN SHELL” after CD‘ing into the MyFirstMirrorApp directory)

runmyproject.cmd

rem note you must setup mvn before trying to run this project in this way...

start mvn compile os:run dashDmodule=mirror

echo sleep
ping 1.1.1.1 -n 1 -w 60000 > NUL

start mvn compile os:run -Dcluster="total_members=2,1" -Dmodule=processor

echo sleep
ping 1.1.1.1 -n 1 -w 60000 > NUL

start mvn compile os:run -Dmodule=feeder

Once everything is running, I can start the GigaSpaces management User Interface by executing: the following from within my “MAVEN SHELL“

>start gs-ui

Now comes the time for the final script of the day:

I store this script in the tools/maven directory.
This is once again executed from within the same “MAVEN SHELL” that I have been using up until now.
This script starts up the HSQL Viewer and connects to the database being used by the mirror service to store data:

lookatmydb.cmd
mvn os:hsql-ui
dashDdriver=org.hsqldb.jdbcDriver
dashDurl=jdbc:hsqldb:hsql://localhost/testDB

executing the following query while in that viewer shows the data stored there :

select * from PUBLIC.DATA

Edge High Performance Computing

Owen Taylor — Wed, 27 Aug 2008 19:57:01 +0000

I ran across this excellent article on Edge HPC and by golly, it sounds like the need for both of the following is growing at a huge rate:

Increasing the efficiency of the software solutions made to solve business problems
Providing adaptability of software systems to changes such as increased load and failures

They call this new landscape: “Edge High Performance Computing“, as opposed to “High Performance Computing“

To my mind the major difference here between these classes of problems is the need to maintain state and the capacity to leverage less-exotic hardware and more “modern” and likely Object-Oriented software languages such as .NET, C++ and Java.

I like very much the quality of the content of the Tabor research work and look forward to future updates from them regarding these topics.

Owen.

Amazon found every 100ms of latency cost them 1% in sales.

Jim Liddle — Wed, 27 Aug 2008 12:41:52 +0000

The GigaSpaces CTO, Nati Shalom, dropped me a line recently pointing out a recent article on High Scalability, which is very interesting. There is an excerpt from the article below:

Latency matters. Amazon found every 100ms of latency cost them 1% in sales. Google found an extra .5 seconds in search page generation time dropped traffic by 20%. A broker could lose $4 million in revenues per millisecond if their electronic trading platform is 5 milliseconds behind the competition.

The Amazon results were reported by Greg Linden in his presentation Make Data Useful. In one of Greg’s slides Google VP Marissa Mayer, in reference to the Google results, is quoted as saying “Users really respond to speed.” And everyone wants responsive users. Ka-ching! People hate waiting and they’re repulsed by seemingly small delays.

The less interactive a site becomes the more likely users are to click away and do something else. Latency is the mother of interactivity. Though it’s possible through various UI techniques to make pages subjectively feel faster, slow sites generally lead to higher customer defection rates, which lead to lower conversation rates, which results in lower sales. Yet for some reason latency isn’t a topic talked a lot about for web apps. We talk a lot about about building high-capacity sites, but very little about how to build low-latency sites. We apparently do so at the expense of our immortal bottom line.

I wondered if latency went to zero if sales would be infinite? But alas, as Dan Pritchett says, Latency Exists, Cope!. So we can’t hide the “latency problem” by appointing a Latency Czar to conduct a nice little war on latency. Instead, we need to learn how to minimize and manage latency. It turns out a lot of problems are better solved that way.

How do we recover that which is most meaningful–sales–and build low-latency systems?

Well, the answer to that questions is “you choose product that are built to handle your latency requirements whilst still allowing you to support scale”. Again it’s clear that Tier Based Architectures with their mish-mash of separate cluster implementations are not only state bound at each tier but are also complex to manage and maintain. GigaSpaces has been talking about these things for a long time now and it’s good to see that there are more general debates and hard evidence of the affects of not building your systems this way.

Why pure caching or compute grids are not enough

Nati Shalom — Sun, 24 Aug 2008 10:22:58 +0000

I came across an interesting comment in our forum from one of our users:I'm doing a PoC for use GigaSpaces in our applications, to have one complete solution, instead of using other distributed cache & computing. Also I'm hoping to...

GigaSpaces XAP 6.5/6.6 new releases

Nati Shalom — Mon, 18 Aug 2008 05:45:39 +0000

GigaSpaces 6.5 was released at the end of June, and we are now working on the 6.6 release, with the first milestone already publicly available. These are major milestones in a series of upcoming releases all aimed at strengthening our...

Video is worth a million words. . . Latest GigaSpacesXAP demo

Owen Taylor — Tue, 12 Aug 2008 21:28:09 +0000

With my wife and kids away, I am having a geeky bachelor time that many who read this would probably find quite thrilling. For example: I spent this last weekend producing a new MonteCarlo application example for GigaSpaces.

It is rich in the patterns it utilizes:

SBA (Colocated spaces with workers)
Master-worker (remote workers which scale dynamically when the workload is too high)
Spring Remoting on top of GigaSpaces – the aggregation work is achieved utilizing our Sync Remoting

The app goes like this:

Historical information regarding several funds is used to predict the future behavior of those funds over a 20 year time period with Minute by Minute fluctuation in prices being calculated and applied.
These funds are grouped in different formations into Portfolios and the entire Portfolio assessed for its growth over the same 20 year period.

In the end, the best combination of funds within a portfolio is determined and reported and the best and worst behaving funds are also shown.

Technically, several portfolios and their associated funds are written as tasks into the system for analysis – the logic needed to process them and their historical information is sent along with them as well. Workers take the tasks, process the associated logic and return a result showing the outcome of the application of the expected variation in price for each fund in the portfolio over the designated 20 year period.

This is performed over a set of possible portfolios – each containing different variations of funds.
Each set is repeatedly simulated to avoid undo skewing and allow for better control of the randomness.

In the end, the client requests a summary of all the results which is created by a service that operates in parallel across the available spaces to take all the results and sort them and aggregate them to determine the most common, best [highest value portfolio], and worst [lowest value portfolio]

This information is returned to the client and printed out to the console.

This morning, I produced a nifty 8 minute flash presentation showing my Monte Carlo application being executed on the GigaSpaces platform.

NB: To keep the video short, and because I only used my laptop, I kept the size of the sets and the iterations at very low levels. Nevertheless, the application scales very nicely and provides fault-tolerance as well.

You can check out the 8 minute video using this link: THE MONTECARLO VIDEO

[you may have to wait a while for the video to load, if it stops
mid-way, play it again from the beginning after it loads completely]

I hope you enjoy watching the fruits of my labors – what could be a better way to spend a bachelor weekend eh?

Cheers,

Owen.

Bridging the gap between the clouds

Owen Taylor — Sat, 09 Aug 2008 15:13:29 +0000

Dekel Tankel of GigaSpaces spoke recently to a hip cloud crowd regarding the risks associated with moving an application to the cloud or grid environment.

Without a GigaSpaces Space-based architecture (what I refer to as a TPC Architecture ) applications running in a cloud suffer from an inability to scale and start to erode the benefits of the cloud. In addition there are questions regarding the ability to maintain state reliably in a cloud: especially a pay by the drink cloud where it can get expensive both in terms of $$ and time to utilize the persistence mechanisms provided by the vendor.

Dekel speaks about and demonstrates how GigaSpaces makes it possible to manage your application services -moving them from machine to machine as needed and also addresses scaling the application by spreading it out [partitioning it while removing the traditional bottlenecks of the database and network boundaries between services]

video

He does a nice job – way to go Dekel!

Cheers,

Owen.

Video Interview on Virtualization.com

Geva Perry — Thu, 07 Aug 2008 15:37:31 +0000

Toon Vanagt at Virtualization.com recorded a video interview with me at Structure 08 in San Francisco (I spoke on a panel). I talk about cloud computing and GigaSpaces, virtual middleware, the economics of pay-per-use pricing and more. Here it is:

Scaling-Out Web Application with GigaSpaces XAP. Part I

Hanan Ben-Lulu — Thu, 07 Aug 2008 14:04:17 +0000

Overview Building a reliable, automatic scaled-out infrastructure for your web application is not a job for the faint of heart. It often involves deep knowledge of your application servers or getting expert consulting. Sometimes both. GigaSpaces XAP application platform will let you do that, and more, with just a few easy steps and at a much [...]