• Home
• About

Archive for May, 2008

Next Page »

Realtime Aggregations

OpenSpaces Developer Challenge WINNERS

Hurray and congrats to the 3 winners of our developer contest OpenSpaces Developer Challenge!
And the winners are (sorted alphabetically)…

Jason Carreira with DomainProxy
Kirill Ishanov with Gigapult
Leonardo Goncalves with GoDo - Goods Donation System

Who won the grand prize?
How was the review/judging process?
What is so special about the winning projects?
How do I join the next contest?

And the Winners [...]

GigaSpaces Blog - New Version and Location

With the recent move of both gigaspaces.com and openspaces.org to a new server, we've now also moved our team blog (which you are now reading) to the new server, running on the latest version of WordPress (2.5.1). Although the blog is still accessible from www.gigaspacesblog.com, the official url is now blog.gigaspaces.com.
We soon [...]

Deployment Predictability

My colleague Uri raised an interesting point in his post. I completely agree with Uri, and would like to give an example.

I have been involved in a project for a mobile operator in the UK during the second half of last year. We built a scale-out SOA activation platform for a new mobile device launch using GigaSpaces. The GigaSpaces platform replaced an existing system.

The original system was built using JBoss as a backend server. Predictions were for a huge increase in activation-requests the system had to handle due to the new device launch. While the system worked fine using the JBoss as a platform, there was no way these guys could predict how many instances of JBoss they would need to run in order to cope with anticipated load. They started to do some benchmarking and performance testing to figure out where the system’s limits were, but they soon found out, that the process was leading them nowhere. This is mainly because the JBoss’s were inconsistent and they hit the scalability ceiling using a few (very few…) nodes. When adding more instances, the overhead of synchronising the JBoss cluster grow exponentially as suggested by Ahmdal’s Law, so the gain in TP that each instance added varied depending on the cluster size and other nodes’ load, which kills predictability all together.

JBoss is just an example in this case. It’s not a JBoss specific flaw, but rather a tier based approached which imposes a limited architecture.

They then came to us to resolves the predictability challenge.

We did an exercise to figure out how the deployment would look like using GigaSpaces, and came up with a linear formula of the HW and number of instances needed to support the given load. More than that, they knew that if the business predictions had been pessimistic, supporting extra load would mean simply deploying more spaces… On top of that, their back office systems did not support HA, and would explode if load increased suddenly, so GigaSpaces also provided HA and throttling for the backend servers. During one overnight test the database failed for about 4 hours, and the system was fully functional and completed users’ requests, while completed requests waited for the database to be brought up again to complete the archiving process. The customer was truly impressed!

Needless to say that the launch went flawless, and there were no issues what so ever with the GigaSpaces based system.

So, yes – Uri makes a good point. GigaSpaces’ customers can predict and properly plan ahead the deployment needed to support their business.

-Guy

GigaSpaces 6.0.5

We just released GigaSpaces 6.0.5, our fifth Service Pack.
The release addresses some critical issues detected over the last couple of months that are related to failover and the Service Grid (see the release note snapshot at the bottom of this post).
The release is available for all version of the product, namely XAP, EDG and the [...]

Let’s play…

I have been involved in a nice gaming project in the last couple of weeks.

In the first meeting, giving an overview, the prospect presented their challenge:

“We store game data in memory as we need very fast response time, and we are reaching our limits in terms of supporting our business. We are launching games which support many thousands of users and we are not able to store a game’s data in a single server.

We tried using a database and running our aggregations against it. The response time we got was not acceptable. We want to store game’s data across multiple JVM, but how will which JVM stores which pieces of the data?”

Well, aggregations and data partitioning go very well together if you have the right technology in hand. Nati Shalom wrote a good post on this subject.

The interesting thing is that GigaSpaces supports aggregation since way before the Map/Reduce term was even thought of… When using JavaSpaces’ readMultiple API against a clustered space, the read request is broadcasted to all the cluster members, who execute the request (locally) in parallel, the client side receives back an array of responses which it then needs to aggregate (Reduce). With OpenSpaces, we have Springified this and it is also supported through Spring Remoting with a pluggable reducer at the client side.

A couple of hours after the dev was kicked off, we were able to demo how a single game can span multiple JVMs, aggregate the game’s data in a simple fashion, and display the aggregated result. The system, which was limited to x10,000 concurrent users, can now scale to x100,000 and x1,000,000 linearly using GigaSpaces.

Bingo!

-Guy

GigaSpaces and partners - new offerings at the SIFMA conference, NY, June 10-12

If you are one of the 7000+ people planning to attend the 28th SIFMA Technology Management Conference in NY next month, make sure to stop by the GigaSpaces booth. We'll be launching and demonstrating exciting new offerings with Microsoft, Lab 49, Amazon, and more. 
You'll find us at the Microsoft mega-booth, in the Grand Ballroom. This year GigaSpaces, along with 4 select partners, [...]

Challenges with scaling MySql on Amazon EC2 and S3

During the past few months I've talked to many Amazon EC2 and S3 users. They all face the same interesting challenge: how do I develop a scalable, transactional and data-intensive application in the EC2-S3 cloud environment?
They are experiencing first-hand the painful gap between scalable hardware (what Amazon provides) and a scalable application. Some of them [...]

Twitter as a scalability case study

So once again Twitter was down last week for a good chunk of time. A lot has been said already about Twitter’s scalability issues. Many have given Twitter as an anti-pattern of how not to deal with scalability and have…

Thoughts on CommunityOne and EC2

Last week I’ve attended CommunityOne at San Francisco. The company I work for (GigaSpaces) had a booth at the startup camp.I’ve recently posted in our company blog describing what I was about to show, and now I would like to share a more “intimate” fee…

About Predictability and Traceability

In this post I will discuss two somewhat overlooked benefits of SBA.But first I want to explain what triggered me to write it. A few weeks ago I read a post based on a book by Daniel Schacter, a Psychology Professor at the Harvard University. Accordi…

GigaSpaces VB API Beta Release

GigaSpaces VB API version 1.0 is now available under OpenSpaces.org.

Up and above some enhancements and stability fixes, the main addition in this release is native notification support for VBA applications.

The distribution includes a comprehensive example of notifications, which includes also (as a side effect) interoperability with Java. It is a simulation of FXSpot Pricing System in which prices are updated by an external system (the Java feeder in our case), and at the other end there is a spreadsheet which is listening to the FXSpot updates.

Here are the basics of the example:

FXSpot POJO:

@SpaceClass(persist = false, fifo = false, replicate = false)

public class FXSpot {

    private String fxName;

 private double modified;

 private float value;    

 @SpaceId(autoGenerate = false)

 public String getFxName() {

  return fxName;

 }

 public void setFxName(String _fxName) {

  fxName = _fxName;

 }

 @SpaceProperty(nullValue = “0″)

 public double getModified() {

  return modified;

 }

 public void setModified(double _modified) {

  modified = _modified;

 }

 @SpaceProperty(nullValue = “0″)

 public float getValue() {

  return value;

 }

 public void setValue(float _value) {

  value = _value;

 }

}

The counterpart PONO (I have omitted the COM methods):

namespace com.mycompany.trading.fx

{

    [ClassInterface(ClassInterfaceType.AutoDual), ComVisible(true)]

    public class FXSpot

    {        

        private string _fxName;

        private double _modified;

        private float _value;

   public FXSpot()

        {

            _fxName     = null;

            _modified   = 0;

            _value      = 0;

        }

        [SpaceID(AutoGenerate = false)]

        [SpaceProperty(AliasName = "fxName", NullValue = "")]

        public string FXName

        {

            get { return _fxName; }

            set { _fxName = value; }

        } 

        [SpaceProperty(AliasName = "modified", NullValue = 0)]

        public double Modified

        {

            get { return _modified; }

            set { _modified = value; }

        } 

        [SpaceProperty(AliasName = "value", NullValue = 0)]

        public float Value

        {

            get { return _value; }

            set { _value = value; }

        }

    }

}

Java feeder – After looking up the space, and initializing Random helpers and currency names, it looks like:

while(true)

{

    FXSpot spot = new FXSpot();

    spot.setFxName(fxNames[ccyRand.nextInt(10)]); 

    double modified = (double)System.currentTimeMillis(); 

    // convert to Excel date

    modified = (modified + 7200000) / 86400000 + 25569;

    spot.setModified(modified);

    spot.setValue(valuesRandom.nextFloat());

    System.out.println(“Writing spot = “ + spot); 

    spaceProxy.write(spot, null, Lease.FOREVER);

    Thread.sleep(sleepTime);

}

Up until now, nothing new really… Now begins the interesting bit!

After omitting the initialization parts of the VBA code, what we have is the following:

Registration function:

Function RegisterNotifications(ByVal sRng As String)

    If GigaSpace Is Nothing Then

        Call Init

    End If

    Dim rng As range

    rng = range(sRng)

    Dim Template As FXObjects.FXSpot

    Template = New FXObjects.FXSpot

    Template.fxName = rng.Value

    Dim modifiers As Integer

    modifiers = NotificationModifiers.Update 

    Call GigaSpace.RegisterNotification(Template, “Notify”, modifiers, sRng)

End Function

It is very easy to explain the code. If you have ever read the docs or used GigaSpaces’ notifications API, you should feel pretty comfortable here. Same rules apply… If you haven’t, a quick browse here will help understanding the API and terminology better.

You simply invoke GigaSpace.RegisterNotification method with a template to match, the name of your notify function, notify modifiers (take, write, update, all), and last argument which is not part of GigaSpaces notify API is called vbHint. It can hold any object the user wishes to receive back when the notify function is triggered for this specific registration (i.e. template match and operation match). In this example I have iterated through a list of currencies, and I pass the cell address of the specific currency to the registration so that when my Notify method is invoked, I can lookup the relevant cell easily.

My Notify function:

Function Notify(ByVal eventType As Long, ByVal pono As Object, ByVal vbHint As Object)

    Dim sRng As String

    sRng = vbHint

    Dim rng As range

    rng = range(sRng)

    Dim spot As FXObjects.FXSpot

    spot = pono

    rng.Offset(0, 1).Value = spot.Value

    rng.Offset(0, 2).Value = spot.Modified

End Function

Note the Notify signature. When registering to notifications, the notify function must have this signature in terms of the types it takes as arguments. The eventType represents the space operation which triggered the notification, and the pono is the object which matched the registration’s template and triggered the notify.

In this example, I take the values from the notification object, and update the relevant Excel cells (I use the vbHint for that).

The example includes a button to UnRegisterNotifications.

Public Sub UnReg()

    If Not (GigaSpace Is Nothing) Then

        Call GigaSpace.UnRegisterNotifications

    End If

End Sub

The UnRegisterNotifications method removes all registrations for the invoking Excel from the space.

One more thing to note is the termination of the Excel process. As notifications require some daemon threads to run at the background, the Excel needs to invoke GigaSpace.Terminate function which cleans up these threads and frees some resources. Failing to invoke the method could result a hanging Excel process. As a best practice, you should catch the Excel workbook close event and call the terminate method.

Example (at the workbook level):

Private Sub Workbook_BeforeClose(ByVal Cancel As Boolean)

    Call TerminateGigaSpace

End Sub

Which invokes the Terminate method:

Public Function TerminateGigaSpace()

    If Not (GigaSpace Is Nothing) Then

        Call GigaSpace.Terminate

    End If

End Function

The example with inline comments and documentation is packed in the setup. After downloading and completing the installation process, the example Excel can be run from –> Programs –> VB Space API –> Examples –> FXSpot Notify. The example is located under \Examples\FXPrices. Follow the readMe.txt to run and change the example.

Enjoy!

-Guy

Next Page »

Copyright © 2008 The GigaSpaces Blog
GigaThoughts is maintained by GigaSpaces Technologies