NASA JPL develops own cloud ‘brokering’ system

And now we wrap up our conversation about NASA’s JPL moving toward cloud computing.

In our final segment with guests Tom Soderstrom, IT CTO at NASA JPL and Khawaja Shams, senior solution architect at NASA JPL, they give us their final thoughts on the benefits of cloud.

TS: I would say there’s a couple of [benefits]. One is, in our industry we look at something we call the technology readiness level. It starts very early with an abstract idea — level 1 — and then when it’s operational, it becomes level 9. Now . . . we’re thinking about the cloud readiness level, so we’re getting JPL up the curve on this cloud readiness level, and we [had] a JPL cloud day — the first in a series. . . . Our overall goal is to run an application and the storage and the computing wherever it’s most appropriate.

So, the cloud for us gives us a new avenue, a new tier of options.

We’ll have our internal data centers with private clouds, we’ll use [a] community cloud . . . and then the ultimate goal is to [use] a public cloud. We have data in Amazon and Microsoft. We also have data in Google’s cloud.

To do that, we need some kind of cloud brokering, and we went out to industry and tried to buy it, frankly, but it doesn’t exist yet, so we’re creating it. We call it the Cloud Application Suitability Matrix — CASM — and that’s the set of questions that gives a score and assesses in which cloud this particular application is the most suitable to run. We think that’s going to be a big trend — this cloud brokering, if you will.

The partnering part, I can’t stress enough, how important it is for all of us in government and the private sector to just get started — to try it — because you learn a lot.

One unanticipated consequence is, of course, there’s a lot of excitement about the cloud, so you’re making connections and you’re making partnerships that otherwise would have taken a lot longer. We have very good relationships with lots of vendors and agencies.

The last piece, I would say, is . . . the CIO at JPL came up with this idea of replacing the procurement screen with a provisioning screen. That kind of says it all. We’re trying to give self-service to the users of IT so that they can get the computing they need when they need it, and turn it off when they need it, so we can spend less money on IT and more money on science.

The whole effort is to keep it real, and we did that from the very beginning and it’s proven very effective. It’s not an IT benefit, it’s a business of the institution benefit.

KS: One thing I’d like to add is, I know that a lot of institutions are very wary of security.

At JPL, instead of stopping to use the cloud because of security problems, we are trying to address the security problems and trying to create best practices and secure ways ot use the cloud without actually compromising the privacy or integrity of our data.

Our admission developers are working very closely with our office of the CIO and the IT security teams to make sure that we can leverage the benfits of the cloud without compromising our security.

TS: We think that the cloud could be more secure than what we do today, because it becomes, in many ways, more uniform so you can react to threats much more quickly and you can segment off things like denial of service attacks and keep going in a different part of the cloud. We have worked very closely with key vendors and cloud security teams . . . and the biggest obstacle, I would say, is going to come from the auditing function.

The auditing function needs to figure out how an application that used to run on one server in one data center now could [run] on multiple servers in multiple data centers. How do you audit that to make sure it’s secure? Until we can do that, we probably can’t go live with anything substantial.

So we’re working very closely with vendors and the auditors to facilitate that, be an early explorer and help industry in that area.

NASA JPL crowdsources with cloud

We continue our conversation about NASA’s JPL moving into the cloud.

Today, we start off discussing how President Barack Obama’s Open Government Initiative is influencing cloud at agencies — and whether or not cloud is helping JPL to comply.

Tom Soderstrom, the IT CTO at NASA JPL and Khawaja Shams, senior solution architect at NASA JPL tell us more.

TS: Essentially you can divide what we do in two ways. One is, it’s good for the mission. It makes us do better science. The other one is about communicating that to the public and getting the public excited. Our stockholders are the public. If the public wants to know more about space and science . . . it will go through the budget.

We’re very pleased to see that it’s a cloud and we’re big supporters of data.gov. We think it’s a fantastic idea — [where] you can get the data out at less cost and much more easily to the scientists and the public. So, we came up with this term . . . of citizen scientists. If they could get access to the data much more easily and quicker, they could maybe even help turn the wheels of science.

We worked with Microsoft using their Azure cloud on [a project] called Be a Martian. Citizens are able to do anything from tagging images online to creating programs that tag the images online. It’s a contest and . . . it’s been very successful. It’s a way of crowdsourcing and we took the images — 250,000 images from the Mars rovers Spirit and Opportunity — put them in the Azure cloud and gave the citizens access to it. It’s been terrific. We did the same thing with at EclipseCon.

KS: EclipseCon is a developer conference. Roughly a thousand or so developers worldwide attend each year in Santa Clara, and we held a contest there called the e4 Rover contest where we allowed developers to drive a mini robot around a Martian landscape basically that we had put together. We used this as an opportunity for public outreach, as well as to get developers to build interfaces to command the robot and view the telemetry that is coming back from the robot.

In order to run this contest, we needed a lot of infrastructure that we didn’t want to just go buy for this one week contest. So, we ran the entire contest on the Amazon cloud and leveraged a lot of the services that are very common to companies like Amazon and Microsoft and Google and we were able to get these for free and very quickly — services like load balancing . . . [and] getting computers running in multiple data centers, and services like the delivery of images to the operators that were, in this case, the developers. This project was actually quite successful and it made venues like Slashdot and Digg.

We ended up getting a lot of open source code back that we can go ahead and directly use to make very useable interfaces.

TS: What surprised us a little bit was the quality of the code that these developers came up with during the conference. It was 24 by 7 and Khawaja was there manning it, and the ways of lighting the road that the developers came up with were quite ingenious, including one on an iPhone. So, the crowdsourcing works both ways and we are quite excited about it.

NASA’s JPL uses cloud so you can see Saturn

NASA’s Jet Propulsion Laboratory is moving into the cloud.

Today we bring you the first segment of our three-part chat with Tom Soderstrom, the IT CTO at NASA JPL and Khawaja Shams, senior solution architect at NASA JPL.

We start off by asking the obvious — why cloud, and why now?

TS: What we did is, about two years ago, we looked at the industry trends that were coming up, and we came up with seven — and cloud computing was one of them. JPL does early exploration of technologies to see if they can benefit the scientists and technologists. So, we started looking at it two years ago and it became very relevant, because we have a tremendous increase in demand in IT for the mission.

We have the busiest period ever coming up here for JPL within the next two years, so we need more computing, more storage, more data centers, and much more flexibility and speed. We want to run and store data and programs wherever it’s most appropriate, and the cloud looks like it would have that potential for us. So, that’s why we started looking at it.

KS: That was basically our reasoning, and then we also realized that a lot of our missions and scientists had very elastic needs for computing . . . where they would need a lot of computation for a few hours in a day, and then their demand would shrink down to basically a very few number of computers for the rest of the day. We found that the cloud can offer very economical options to leverage elasticity.

FCB: You are out in California. We’re here in D.C. Have you gotten any specific guidance or have you shared best practices with NASA [out here]? We recently spoke with NASA CIO Linda Cureton, and she just talked generally about what she’s doing at NASA here in D.C. When it comes to cloud computing. Have you shared best practices . . . or, if not, do you plan on doing that in the future?

TS: Yes, indeed. We are sharing. In fact, our whole approach is about being proactive and exploring multiple clouds and share our lessons learned — the good and the bad — so we can advance the state of the practice, but also so we can learn and not make big mistakes. Linda Cureton is a believer in cloud, and so is my boss Jim Rinaldi, CIO of JPL. We share best practices with all the NASA centers, but also with other industry players, including internationally. We’re working with ESA, the European Space Agency, to try to see how we could collaborate better. So, we share best practices and we share our vision and our approach, as well as the results and road map. That way we can come to a better cloud readiness level.

FCB: Overall, what are the benefits of cloud computing for JPL. The only reason we ask is because we’ve talked to other agencies . . . and some are a little bit wary so far. . . . How did you make the decision with cloud and what are the benefits?

TS: That’s really the crux of the matter for us. . . . Two years ago we looked at the cloud and saw it had all of these potential benefits, and we worked a lot with Computer Science Corporation (CSC) and we studied the cloud. So, we saw, in depth, what the potential benefits were and the potential drawbacks were, but every institution is different. What we wanted to do was to keep it real for our scientists and technologists. We had to get in and explore early.

We are on the other side of the coin, I would say . . . [because] we believe in proactive, early exploration so we can understand what the net benefits are for JPL and our personnel, as well as for NASA. We can’t do that unless we’re in there trying it. That was our overall approach — to just get started and to try it. . . . Two years later, I think we have a pretty good handle on what the real net benefits will be and are, and what the real drawbacks are. We are seeing some real benefits.

The big issue was security. Could we actually use the cloud right away in a secure way?

What we did was come up with something we call the ‘wheel of Security’. We divided our data — or tagged it — and asked, is it public data, secure data and is this something we can do today? When we did that, we discovered that we have a lot of public data and we can do our processing [in the cloud] both to benefit from it immediately, but also to learn.

Now what we’re doing is looking more at the private data, if you will and this wheel of security — we’ll talk to the vendors and providers — we’ll spin it and when they are ready to take [it] on, we’ll [move] the next big chunk.

KS: We had to process 184,000 images of Saturn for a public outreach project, and we started processing it in our labs and after a couple of weeks we decided it was going to take too long. We were only using one machine to run it. We put it out on the cloud and we were able to process the same images in about 5 hours.

So, the dramatic increase of 15 days to 5 hours is very worthwhile as a strategic research advantage. One of the things that folks have pointed out is, if you have a computation that requires 100 hours of computation, if you run it on one machine for 100 hours or 100 machines for one hour, it literally costs you the same thing, except you gain 99 hours of more research time. We want to bring this benefit down to our scientists.

We have already some of these successes come through with our public data processing.