The UberCloud Experiment started in July 2012, with a discussion between Burak Yenier and Wolfgang Gentzsch about cloud adoption in technical computing. Remember, the first cloud infrastructure – Amazon’s AWS – was just six years old, and an “Early Majority” of enterprise and scientific users started using it for some of their applications. But many engineers (in fact all of the engineers we talked to, then) were very reluctant to test the cloud for their highly complex engineering application simulations; they simply did not believe that the cloud could be a powerful tool for them to accelerate their simulations and allowed them to develop higher-quality products with cloud, and they had good reasons for their distrust.
At that time, we identified a long list of cloud computing challenges especially for high-performance computing applications, in areas like manufacturing, energy, life sciences, finance, and more, and we decided to explore these further, hands-on, and in June 2012 [01], the idea of the UberCloud Experiment was born. We invited engineers, computing resource providers, software providers, and HPC experts from all over the world to work with us on moving the engineers’ applications to the cloud, and we hired and trained 10 mentors who helped supported them in their endeavors. Eight months later, after the first 100 cloud experiments and 50 case studies, we published a summary, with lessons learned and recommendations in the Bio-IT Magazine, [02].
Our efforts were paying off. Based on the 230 engineering cloud experiments and the experience gained over the past 11 years, we have now increased the success rate of the individual experiments to 100%, as compared to 40% in 2013 and 60% in 2014. Based on our early experience gained from the cloud experiments in 2012 – 2014, we reached an important milestone when we introduced our unique UberCloud HPC software containers that we have built to host the engineers’ complex simulation workloads. Use of these containers by the engineering teams dramatically improved and shortened cloud implementation times (so-called on-boarding) from an average of three months to just a few days.
We found that enterprises would strongly benefit from technical computing in the cloud. By gaining access on-demand from their desktop workstation browser to additional remote and powerful computing resources, their major benefits now are: the agility gained by immediate access to any computing, storage, and networking resources: by shortening product design cycles through shorter simulation run times; the superior quality achieved by simulating more and more sophisticated applications; and by running many more iterations to look for the best product design. These are benefits that can increase a company’s ability to innovate and compete, dramatically.
But how far away are we today from an ideal HPC cloud platform for engineers and scientists? In the beginning, we did not know. Engineers were just facing challenges like security, privacy, and trust; lack of cloud control; conservative software licensing models; slow data transfer; uncertain cost & ROI; identifying best suited resources; and lack of standardization, transparency, and cloud expertise. However, in the course of the UberCloud Experiments, as we followed each of the (so far) 230 teams closely and monitored their challenges and progress, we have got an excellent insight into these roadblocks, and how our engineering and scientific teams have tackled them, and how we are now able to reduce or even fully resolve them.
If you want to know more about UberCloud's novel Engineering Simulation Platform and unique HPC Containers, please contact us at https://www.theubercloud.com/help.
References:
[01] The Uber-Cloud Experiment, HPCwire. June 28, 2012,
https://www.hpcwire.com/2012/06/28/the_uber-cloud_experiment/
[02] Exploring Computational Biology as a Service: UberCloud Round 3. Bio-IT Magazine, March 2013,