Archive for March, 2008
Scaling Out MySQL
March 29th, 2008With the recent acquisition of MySQL by Sun, there has been talk about the MySQL open source database now becoming relevant to large enterprises, presumably because it now benefits from Sun’s global support, professional services and engineering organizations. In a…
Amazon gets more elastic
March 27th, 2008Amazon EC2 now makes it possible for failure to be less noticeable within EC2 due to the new ability offered to have an IPAddress move from box to box upon failure. This is really cool because it means you can host your webserver as a service within the GigaSpaces AMI and when it automatically fails over due to our service management and self-healing behaviors, the end user will have a hiccup but will be able to refresh and continue without changing the URL!
Another cool new thing EC2 is offering is the ability to host primary and backup nodes in different physical “Zones” - meaning Amazon will guarantee that the backup location is not on the same power grid as the primary. Using GigaSpaces to create and deploy the instances makes it even easier to ensure this desired behavior is achieved.
See Amazon’s Feature Guides for Elastic IP Addresses and Availability Zones.
Learn how to build your first EC2 Applications using GigaSpaces
Cheers,
Owen.
Sharing Results of Expensive Computations
March 25th, 2008A customer recently contacted us to discuss how to use GigaSpaces XAP to minimize the number of times an expensive computation must run. The particular domain is option price stress testing, but the problem is more general and the solution demonstrates a couple of interesting GigaSpaces XAP features.
Consider a service that executes an expensive [...]
Tonight: GigaSpaces Cocktail Event in San Francisco
March 19th, 2008If you missed our cocktail event in Menlo Park last night — you have another chance. We are hosting another cocktail event in San Francisco tonight at 6:00 PM at the W Hotel (181 3rd street, corner of Howard).
Good food and drinks will be served and some good conversation on technology.
If you are in the [...]
Excel That Scales at Microsoft Developer Conference
March 17th, 2008As a Microsoft partner we've been invited to attend the 6th annual Microsoft financial services developer conference which took place in New York City last week.
It has been a great experience (about 900 people attended), and for us at GigaSpaces it was a huge success.
We presented two live demos:
GigaSpaces Excel that Scales: [...]
Scaling out your tier based application in three steps - Now online
March 17th, 2008A while back I wrote Architecture You Always Wondered About: Lessons Learned at Qcon, where i summarized some of the lessons learned from the Amazon, eBay, Yahoo and LinkedIn architectures, which enabled them to address their scalability requirements. During that…
Cloud Aware Classpath
March 12th, 2008One of the most challenging aspects in deploying large distributed systems is making sure the right jar files are located where they are needed. For a cluster that contains many nodes (tens and hudreds and even thousands) having the ability to deploy the application code easily is very important.
This problem is not new and indeed there have beens several solutions for it in the past:
The first one is of shared file-system. All hosts have access to a shared drive. A classppath variable is set to this shared drive and upon change this shared drive is the only place to put new jar files. This simple solution is ok when it comes to static deployments and when we know all hosts are on the same physical LAN. Another drawback of this simple solution is that it represents a SPOF in the system. Also, the singularity imposes a contention point that in many cases becomes a bottleneck. Also, it is imperative to make sure the FS is as available as our SLA requirements. This may become very costly.
The second approach is of a codebase server. This is the approach used by RMI and by JINI. As java class loader is a URLClassLoader, all we make sure is that our jars are behind a reliable HTTP server and when the JVM starts it looks for the classes at the codebase server as well. The codebase server is a first step in virtualization. A url is abstracted by a host name controlled by a DNS and can be changed without impacting the application. There are few implementation problems with this approach: The first is that when it comes to downloading the jars, in real life, this may lead to a network storm which will result in system hickups as jars are downloded fully and in real life systems those jars may be very large. Another problem with the codebase server approach, is that the URLClassLoader behavior is static. Once a class has been loaded into the class loader it can not be changed, and in order to replace it the class loader needs to be destroted, which practically means in many cases, the process needs to be restarted.
What we’ve introducing in 6.5 and is already available in M7, is what we call "Cloud Aware ClassLoader". Once a VM in the cloud requires to load a class that it does not have it’s class defition, it calls to the "cloud" classloader to retreive the class definition. This mechanism is fully integrated with the JVM class loader, in many cases the sequence of loading a class is transitive. For example if class A implements interface I, the classloader starts loading class A and as it idetifies other class dependecies such as interface I are missing, it looks for it in the cloud for the interface’s class defintion.
The full power of this feature unlocks dynamic capabilities of patterns such as MapReduce and Master Worker. Where is such cases the implementation of the Work object (M/W) can be changed at runtime while the system is running without the need to re-deploy the code on all nodes again.
Cosider the following scenario. Task interface is defined with one execute mehtod. Worker implementations resides on cloud nodes know how to consume a Task object and to call it’s execute method. The first client connects to the cloud and submit an implementation of Task which is called TaskA. All he’s doing is connecting to the cloud and submitting his task. No need for task deployment on cloud nodes is required. A second client connects to the cloud as well and submits his own implementation of Task named TaskBTask and he’s not deploying as well. The beauty of it is that if client A would like to be able to use TaskB the class information of TaskB will be shipped automatically to his JVM upon demand without any configuration.
The funny part, is that we didn’t plan in advance to solve this problem in the release, we had other issues to attend. However, as we had several painful experiences in deployment of huge clouds in our lab and on EC2, the Runtime Platform team, decided that they are too spoiled to work with non-elegant solutions, and in few days work, put this great feature into the product.
You’re all encourage to give it a spin and share you’re thoughts on this with us.
Cheers,
Guy
.Net Customer Announcement: Susquehanna (SIG)
March 11th, 2008In the past few months we've made several exciting announcements, such as our partnership with SpringSource , the expansion of our executive team, the launch of our community site OpenSpaces.org , the OpenSpaces Developer Challenge and the Start-Up Program . But there is nothing like a customer announcement, as in today's press release [...]
Moving to Extreme Transactions Processing using Lean methodology
March 3rd, 2008In one of my recent posts I used a production line analogy to describe the applicability of methodologies like “Lean” taken from production line optimization to transactional systems. As a reminder, here’s a good explanation about Lean from Amit Rathore’s…
TradeTech Architecture 2008
March 3rd, 2008Next week, GigaSpaces' CTO Nati Shalom is going to participate in a panel about the future of grid technologies in the TradeTech Architecture conference in London. This conference hosts many top executives, mostly from the biggest players in capital markets. It should be quite interesting, whether or not you are coming from the finance industry.
Some [...]
Underneath the Skin – What is the CPP Processing Unit and how can I build one?
March 1st, 2008I went to visit this extraordinary and shocking exhibition a few weeks ago - Bodies. It is located near pier 17 in Manhattan (not far from South Street Seaport, which I mentioned in my previous blog post).
See the links below for the exhibition site and a few videos: http://www.bodiestheexhibition.com/intro.html http://www.bodiestheexhibition.com/video/bodies-video-broadband-1.wmv http://www.bodiestheexhibition.com/video/bodies-video-broadband-2.wmv
You can find the same [...]
