The GigaSpaces Blog

Startup Camp - San Francisco

Posted 4 May 2008 @ 2:27 pm by Dekel Tankel

Come and join us at the GigaSpaces booth in the Startup Camp event, Moscone Center, San Francisco. We will present a live demo of the GigaSpaces solution for the Amazon Elastic Compute Cloud (“EC2”) . In this demo we illustrate how GigaSpaces XAP, the scale-out application server, bridges the gap between on-demand hardware scalability and on-demand application scalability. We will show how you can quickly spin additional Amazon Machine Images (“AMIs”) as you need them and how your application utilizes this additional capacity and high availability via the GigaSpaces framework. And …you get all of this in a “pay by the drink” model, you don’t use, you don’t pay! See you all there Dekel

Posted in GigaSpaces | Top Of Page |del.icio.us │Digg it|Leave a Comment »

Intro to GigaSpaces Webinar

Posted 27 April 2008 @ 4:02 am by Geva Perry

On Wednesday, April 30, at 12:00 PM EDT, Owen Taylor , famed GigaSpaces evangelist and world-renowned speaker and writer, will be presenting an introduction to GigaSpaces over the Web. The presentation is targeted mainly at developers, but others may find it useful as well. To register and participate go to this link .

As of the writing of this blog post more than 90 people have already signed up.

Posted in GigaSpaces, Events | Top Of Page |del.icio.us │Digg it|Leave a Comment »

Integrating GigaSpaces persistency service into an existing tier based system

Posted 23 April 2008 @ 12:04 pm by Mickey Ohayon

A common issue I’m facing recently is how to integrate existing tier based applications with GigaSpaces persistency service, AKA persistency as a service (Paas) or mirror . The motivation is often a result of the acknowledgment that a standard tier based application fails to scale when facing the database throughput limitation.

Software Caching technologies (overlooking their complexity and maintenance they add to the stack) are useful for read operations. You’ll find out quite fast that operations as insert or update under a transaction require synchronous disk write operations, which are expensive both in performance and cost perspectives. Storage devices (NAS/DAS) together with database clustering software are becoming major investments for large scale applications.

GigaSpaces XAP/EDG provides both caching and persistency services based on its memory virtualization capability. GigaSpaces XAP/EDG simply utilize the available machine’s memory on the cloud (grid computing). Using the space as a fast clustered in memory entity, the transactions are replicated into a minimum of two separated physical machines. The persistency service is an asynchronous (thus out of the latency path) service that stores the data into the disk using the CacheStore interface.

Now comes our out of the box Hibernate CacheStore implementation, where once the mirror service is configured to use Hibernate, it will store the data asynchronously based on the hibernate mapping files (*.hbm.xml) that contain the object relational mapping meta data.

The nice thing about using Hibernate and GigaSpaces is that the combination of the two provides a non-intrusive way to keep your programming model and services as POJOs (plain old java objects) that are abstracted from the caching or disk implementation. This is done by either using annotations or external xml descriptor files .

So after reviewing the theory, I thought you might be interested in a practical example, showing how to integrate GigaSpaces mirror service into an existing tier based application; see below.

Example: Integrating GigaSpaces Mirror Service into an Existing Tier Based Application

The Hibernate mapping implementation is encapsulated in GigaSpaces mirror service, meaning this can be a non intrusive procedure in existing projects. Using hibernate-tools you can auto-generate your Pojo sources and hbm files using existing database tables just by configuring the database connection.

There are 2 options of using hibernate tools:

Using a simple ant + hibernate tools jar file.
Using an up to date eclipse plug-in; more details.

This example uses option one, which requires the download of the workshop toolset; download .

Note: this is an old implementation of hibernate tools, but a very lightweight version, which I find sometimes to be more efficient as appose to play with eclipse plug-ins that might end up with eclipse malfunctioning.

Install & config:

Unzip the workshop_tools tar.gz file into a working directory
Make sure ant is installed (http://ant.apache.org/)
Configure the build.xml to match your JDBC and package settings

Adding a usage target to the build.xml will give you a nice default option menu:

<echo message="clean –> Delete all generated demo files."/>

<echo message="bottomup.middlegen –> Run Middlegen and generate Hibernate mapping files."/>

<echo message="bottomup.hbm2java –> Generate .java from .hbm files."/>

<echo message="gui.hibernate –> Run hibern8ide for ad-hoc queries."/>

</target>

Remember to point usage target as default target:

Running ant should produce the following:

Step one:

Doing some "reverse engineering" by creating hibernate mapping files (hbm.xml) files out of existing database tables

Configure the target using your JDBC properties (set the package to your preference)
Execute Ant bottomup.middlegen command

Update the build.xml jdbc configuration:

<target

name="bottomup.middlegen"

description="Run Middlegen and generate Hibernate mapping files.">

<middlegen

appname="Toolset"

prefsdir="${basedir}"

gui="true"

databaseurl="jdbc:mysql://localhost:3306/test"

driver="com.mysql.jdbc.Driver"

username="root"

password="1234">

<hibernate

destination="${gensrc.dir}"

package="org.hibernate.workshop.toolset"/>

</middlegen>

</target>

Running: ant bottomup.middlegen

Step 2

Select the tables you wish to generate object from and click Generate

Step 3

Note: since this is an old hibernate tools version, you'll need to update the DOCTYPE header to:

<!DOCTYPE hibernate-mapping PUBLIC "-//Hibernate/Hibernate Mapping DTD 3.0//EN"

"http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd">

Step two:

Create the POJOs out of the created hibernate mapping files (see: Step one)

Using Ant bottomup.hbm2java command
Create GigaSpaces mapping files (gs.xml); see instructions
Or use GigaSpaces annotations; see instructions

Step three:

Enabling the mirror service; see details and instructions.

Optional:

Using the hibernate gui to validate the hibernate.config.xml and query the DB:

>ant gui.hibernate:

Step 4

Step 5

If you're interested in some further reading, here is a useful link for Hibernate ORM users.

I'd be happy to hear your thoughts, or try it out and let me know how it worked for you!

Mickey.

Posted in SOA, Java, Application Architecture, Application Performance, GigaSpaces, sba, space-based architecture, JavaSpaces, J2EE, Data Grid, Caching, Hibernate, Development | Top Of Page |del.icio.us │Digg it|Leave a Comment »

GigaSpaces, Microsoft and Lab49 Event: Excel That Scales

Posted 23 April 2008 @ 12:02 pm by Geva Perry

On Wednesday, May 7, 2008, between 8:30 Am and 11:00 AM ET, GigaSpaces, jointly with Microsoft and Lab49 , will be hosting an Executive Breakfast Seminar, entitled Excel that Scales - High Performance Excel Applications for Traders and Analysts in Financial Services. The event will be held at the Microsoft offices in NYC (Radio City Conference Room, 1290 Avenue of the Americas, New York, NY)

Speakers:

Craig Saint-Amour, industry solutions director, capital markets, financial services group at Microsoft
Luke Flemmer, managing director at Lab49
Owen Taylor, director of worldwide technical communications at GigaSpaces

To receive an invitation please contacts merida@gigaspaces.com.

Description of the event:

GigaSpaces Technologies is hosting an executive breakfast seminar along with Microsoft Corporation and Lab49 to discuss the GigaSpaces and Microsoft joint solution, which enhances the scalability of Microsoft Excel . This offering allows enterprises to harness the power of their data centers to scale their Excel-based applications and dramatically improve their users' productivity. Craig Saint-Amour, Luke Flemmer and Owen Taylor will present a complete solution that allows enterprises to off load data and computation (Excel macros, VBA, etc.) from users' desktops to server side, and benefit from improved resiliency, scalability, and total cost of ownership of their Excel based applications..

During the seminar Saint-Amour, Flemmer and Taylor will discuss solution benefits such as how to:

Handle massive amounts of data (e.g. market feeds) in a reliable manner
Dramatically reduce processing times and enable more accurate risk and portfolio management (perform more VAR calculations and Monte Carlo simulation runs per hour)
Share data between multiple traders and analysts, and enable better collaboration
Shield users from desktop and server failures
Enable Excel applications to scale up (and down) to meet varying loads
Free up desktop resources
Effectively migrate Excel logic to run as server side, managed code

Speaker bios:

Craig Saint-Amour, is responsible for strategy, solutions delivery, partner management, marketing and industry sales in the financial services group at Microsoft Corporation. He has more than 30 years of U.S. and International business experience and has focused the last 20 years on the financial services industry. Prior to joining Microsoft, Saint-Amour worked at SUN Microsystems and Digital Equipment Corporation within the Corporate Accounts Program. Additionally, he has had tenures with both Wang Laboratories and IBM.

Luke Flemmer is a founder and managing director at Lab49, working with existing clients overseeing Lab49's engagements, advising them on technology direction, particularly in the area of fixed income. Prior to founding Lab49, Flemmer worked alongside other founding partners at Simile Software, where he managed the engineering team and co-developed the firm's core intellectual property. Additionally, he has served in executive roles at both Braindock and Nimbl Inc. Flemmer has spent most of his career consulting and developing technology for major corporations, and his particular interest and experience are in the areas of real-time, high throughput systems, and in the design and application of evolutionary and adaptive software systems.

Owen Taylor is the director of worldwide technical communications at GigaSpaces Technologies. His areas of expertise include J2EE design patterns, performance tuning of J2EE applications, and translating new architectural concepts and technical capabilities of space-based solutions into readily accessible formats. Prior to GigaSpaces, Taylor presided in executive and consulting positions at Identify Software, The Middleware Company, The New CustomWare Company and Inprise.

Posted in GigaSpaces | Top Of Page |del.icio.us │Digg it|Leave a Comment »

Subscribe to GigaSpaces Podcasts

Posted 23 April 2008 @ 11:33 am by Geva Perry

Owen Taylor, GigaSpaces Sr. Director of Technical Communications, produced a series of podcasts with various folks from the GigaSpaces team. You can find these podcasts on the GigaSpaces web site here . You can also subscribe to them in iTunes by clicking on the Podcast logo below.

The podcasts give you the inside scoop on GigaSpaces vision, plans, product roadmap, development practices, new projects and more.

Check them out .

Posted in GigaSpaces, podcast | Top Of Page |del.icio.us │Digg it|Leave a Comment »

Single mirror multiple data-sources

Posted 15 April 2008 @ 8:10 am by Julien Eluard

As part of my job, I often visit customers of very different needs and application types, but most of them have one thing in common; data, data and more data. Data related challenges are numerous but in this post I’d like to share with you a few alternatives to solving a common scenario; where the application is using multiple persistency layers that need to be synchronized, without compromising consistency or performance.

Persistency as a Service

Data modification performed on a GigaSpaces cluster can be received asynchronously using a central process. This process is called the Mirror Service.

This concept is mostly used in two scenarios:

As a persistence mechanism; click here to read more.
As an optimized over the WAN replication facility

The Mirror Service allows providing your own logic to deal with batches of modifications by implementing the BulkDataPersister interface. While a powerful capability, this does not allow configuring multiple implementations simultaneously.

Simultaneous Configuration

A solution to workaround the challenge stated above is to implement a specific BulkDataPersister that broadcasts the data to concrete implementations, internally. Every method calls will then be forwarded to all persistency layers. Composite BulkDataPersister is a generic implementation of this pattern, allowing to simply configure multiple concrete BulkDataPersister used by a Mirror Service.

What do I mean by simply? All you need to do is to configure your regular BulkDataPersister implementations (using either regular property file or spring injection) and then configure the Mirror Service to use Composite BulkDataPersister.
Internally, a thread pool will be used to delegate the execution of all implementations in parallel.

Inconsistency Handling

The Composite BulkDataPersister approach could leads to inconsistency between the various BulkDataPersisters, in case one of the delegates fails.
To solve this, the following steps can be taken:
• A generic strategy can be used for handling partial failures.
• One of the default strategies will use another space to store all information related to a particular failure.
• Ultimately, if the strategy itself fails, an exception is propagated to the Mirror Service, which then triggers the redo-log mechanism (so one batch might be received several time by a delegate).

Implementation of this pattern with full documentation is available here.

Since inconsistencies pose a potential risk using this approach, I recommend keeping in mind all failure scenarios, including the most unlikely ones. If everything fails, data modifications can be received several times by persistency layers due to the Mirror Service redo-log granularity.

Nevertheless, this simple pattern allows leveraging the performance improvement provided by the Mirror Service in highly distributed and dispersed environments, while still ensuring full reliability. In fact, one of our clients with strong WAN performance requirements is already using it.
Further improvements to this pattern might include transaction usage and JMX management.

Do you consider those features important? Are the available entry points sufficient to your needs?

Posted in GigaSpaces | Top Of Page |del.icio.us │Digg it|Leave a Comment »

MuleCon: Scaling Mule Across a Grid

Posted 1 April 2008 @ 5:15 am by Geva Perry

For those of you in San Francisco today and tomorrow (April 1-2), and especially if you were planning on attending MuleCon , please come see Uri Cohen of GigaSpaces present about Scaling Mule within a Grid. In his presentation, Uri will show how you can scale-out Mule, with fault-tolerance and high-performance, across a cloud of machines.

MuleCon, as you can guess, is the annual conference for Mule users organized by the people behind Mule, MuleSource . At GigaSpaces, Uri Cohen and others have been working on a very tight Mule integration. It will actually be part of GigaSpaces XAP in version 6.5. Although not in GA, you can download it now . Also check out the online documentation in our award-winning Wiki.

Uri will be speaking on Wednesday, April 2, during the afternoon breakout session at around 3:00 PM in the Grand Ballroom of the Hilton San Francisco Financial District on 750 Kearny St.

Hope to see you there.

Posted in GigaSpaces | Top Of Page |del.icio.us │Digg it|Leave a Comment »

Sharing Results of Expensive Computations

Posted 25 March 2008 @ 9:51 am by Patrick May

A customer recently contacted us to discuss how to use GigaSpaces XAP to minimize the number of times an expensive computation must run. The particular domain is option price stress testing, but the problem is more general and the solution demonstrates a couple of interesting GigaSpaces XAP features.

Consider a service that executes an expensive computation. In this case, “expensive” means that it takes a significant amount of time to process a request. Many clients will be making requests nearly concurrently. Some requests from different clients will be identical. The result for a particular request is valid through the end of the day.

The system is configured to have a number of instances of the service running simultaneously. Communication with the service instances is via the the space: clients write requests to the space and retrieve the results when they are available. Either blocking read or notification can be used to get the result.

The primary requirement of this system is that the service should not be invoked unnecessarily. This means that results, once computed, should be available for as long as they are valid and that requests that arrive while an identical request is being processed should not cause a second service invocation.

An additional requirement is that any mechanism for meeting the processing requirements should be transparent to the clients. Clients should simply write requests and read results.

The first requirement, that results should be available for as long as they are valid, is trivially easy with GigaSpaces XAP. When the service computes a result, it writes it into the space with a lease of the appropriate length. From then on the result is available immediately to any client making an identical request.

The second requirement, not processing identical requests even when the result is not yet available, is more technically interesting. The basic technique, unsurprisingly, uses a mutex. When a request is written to the space, it is picked up by a worker that is responsible for invoking the service. Before that invocation, the worker checks for an exiting result. If one exists, the worker moves on to another request. If no result is found, the worker attempts to write a mutex, based on the unique identifiers of the request, into the space. If an identical mutex already exists, the worker moves on to another request. If the mutex write succeeds, the worker invokes the service and, when it completes, writes the result to the space. That is:

  take request
  if no result exists
    try to write mutex
    if write succeeds
      invoke service
      write result

The operations involved in taking the request from the space and writing the result are executed in the same transactional context. If the worker fails for any reason, the transaction will abort and the request will reappear in the space.

The implementation of the mutex takes advantage of GigaSpaces’ unique UID to cause the write to fail if an identical mutex is already in the space.

If the worker (or the machine on which the worker is running) fails after writing the mutex but before invoking the service, processing of the request could block indefinitely. Leasing, the second technically interesting GigaSpaces XAP feature in this solution, addresses this problem.

Entries in the space, like many components of the GigaSpaces XAP Space Based Architecture (SBA) have an associated lease. This is the period of time that the space agrees to hold the entry. When the lease expires, the entry is removed from the space.

Leases can be renewed before they expire. This is the feature that allows a mutex to be maintained only so long as the worker that wrote it is running. The worker first writes the mutex with a lease considerably shorter than the expected service execution time. It then creates a lease renewal manager to renew the lease periodically while the service is executing. If the worker or service fail for any reason, the mutex’ lease will not be renewed and it will disappear from the space. This will allow another worker to process the request.

To eliminate the possibility of a worker not finding a result that is about to be written and therefore invoking the service unnecessarily, when the result in written to the space, the mutex’ lease is updated to overlap slightly with the lifetime of the result.

This description is almost as long as the fully documented code, which is available at http://www.openspaces.org/display/SCN/Shared+Computation.

It’s worth noting that this technique, along with many others such as traditional master-worker, map-reduce, implicit workflow, and simple distributed caching, is just one straightforward use case of an SBA supported by GigaSpaces XAP. These techniques can be mixed and matched as appropriate for the solution being developed, all the while taking advantage of GigaSpaces XAP to address enterprise non-functional requirements, including:

Scalability (near linear across thousands of nodes)
Resiliency (self-healing, high availability)
Management and monitoring
SLA driven dynamic deployment
Performance
- extremely low latency
- high throughput

Posted in Application Architecture, Application Performance, GigaSpaces, sba, space-based architecture, Development, OpenSpaces | Top Of Page |del.icio.us │Digg it|Leave a Comment »

Tonight: GigaSpaces Cocktail Event in San Francisco

Posted 19 March 2008 @ 1:31 pm by Geva Perry

If you missed our cocktail event in Menlo Park last night — you have another chance. We are hosting another cocktail event in San Francisco tonight at 6:00 PM at the W Hotel (181 3rd street, corner of Howard).

Good food and drinks will be served and some good conversation on technology.

If you are in the area and would like to attend, please send an email to Roseann Bonnar — roseann at gigaspaces dot com.

Hope to see you there!

Posted in GigaSpaces | Top Of Page |del.icio.us │Digg it|Leave a Comment »

Excel That Scales at Microsoft Developer Conference

Posted 17 March 2008 @ 9:13 am by Alon Lahav

As a Microsoft partner we've been invited to attend the 6th annual Microsoft financial services developer conference which took place in New York City last week.

It has been a great experience (about 900 people attended), and for us at GigaSpaces it was a huge success.

We presented two live demos:

GigaSpaces Excel that Scales: Demonstrate Microsoft Excel as a rich client and GigaSpaces XAP which leverages an in-memory data grid to ensure fast access to critical data; co-locates data and business logic to maximize performance; offers automatic failover and high availability; scales linearly on commodity hardware. - Also known by the people who saw it "Excel on steroids".

"This solution will be applicable to just about everyone who is using Excel and has a data challenge, because the trends in escalation of data volumes and complexity of algorithms used to do calculations is not reversible." as Stevan Vidich, an industry architect for Microsoft Financial Services, is quoted in the Live by Excel, Die by Excel article in Gridtoday.
GigaSpaces WCCS (Windows Compute Cluster): Demonstrate a winning combination of Excel, GigaSpaces and Microsoft HPC solution.

This was by far the best event I have ever attended; it was very focused, with a wide attendance of decision makers and developers that understand and know their current solutions' limitations and are looking for new and creative ways to rise above those limitations.

The GigaSpaces booth was the most crowded one there and the demos attracted many new prospects (there was even an actual prince among them - GigaSpaces is getting majestic!). The demos were so good that even Microsoft's sales people asked us to install our demo on their personal machines.

But the best of all was the "Gigarita", the scale-out margarita that was served during the cocktail hour, everyone loved it!

Look forward to seeing you there next year,

Alon

Posted in GigaSpaces, .Net, Excel | Top Of Page |del.icio.us │Digg it|Leave a Comment »

.Net Customer Announcement: Susquehanna (SIG)

Posted 11 March 2008 @ 3:22 pm by Geva Perry

In the past few months we've made several exciting announcements, such as our partnership with SpringSource , the expansion of our executive team, the launch of our community site OpenSpaces.org , the OpenSpaces Developer Challenge and the Start-Up Program . But there is nothing like a customer announcement, as in today's press release about GigaSpaces customer Susquehanna International Group (SIG).

Every vendor out there, particularly the smaller ones, knows that it's never easy to get customers to agree to these public endorsements. Ironically, it is even more difficult when your product is viewed as a strategic platform that creates competitive advantage and significant economic benefits. And customers definitely view GigaSpaces as a strategic technology. So it's very gratifying when a customer such as SIG agrees to this. Thank you, folks!

Now while SIG is not exactly a household name, it is a remarkable organization and one of the finest financial institutions in the world. Few companies on the globe deal with the volumes of data and transactions that SIG does. So again, it's exciting when they choose GigaSpaces as a core application server to handle these Extreme Transaction Processing (XTP) volumes.

The final aspect that gets me excited about this announcement is that SIG is a pure .Net shop. Those of you who are familiar with GigaSpaces know that our origins (and in fact our product) are Java-based. However, it has been part of our philosophy that we are a "run-time" company. We are judged on having the most scalable, highest-performing, most reliable execution platform in the world, not on the API. We are committed to supporting any popular development framework out there, whether it is a de jure or de facto standard. For that reason we support a variety of Java/JEE standards (JMS , JDBC/SQL , Map/Jcache , JavaSpaces ), the Spring Framework (via OpenSpaces), Dynamic Languages (e.g., JRuby, JavaScript, Groovy), C++ and, of course, .Net . And there are more frameworks and programming models to come. The SIG announcement proves that the model works.

Posted in Java, News, Application Performance, GigaSpaces, JavaSpaces, J2EE, .Net, C++, Spring Framework, OpenSpaces | Top Of Page |del.icio.us │Digg it|Leave a Comment »

TradeTech Architecture 2008

Posted 3 March 2008 @ 6:49 am by Michael Mitrani

Next week, GigaSpaces' CTO Nati Shalom is going to participate in a panel about the future of grid technologies in the TradeTech Architecture conference in London. This conference hosts many top executives, mostly from the biggest players in capital markets. It should be quite interesting, whether or not you are coming from the finance industry.

Some of the issues that will be discussed in the panel are:

Forecasting the impact of rising data volumes and latency demands on future grid configurations
To what extent is the technology set to evolve, and what will be the business implications of this?
Examining the possibility of integrated compute- and data-grids
How long before grid becomes a commodity?

As always, we hope to see you there.

Posted in GigaSpaces, Events | Top Of Page |del.icio.us │Digg it|Leave a Comment »

— Next Page »