Tweet

Follow @petermichaelhan

cloud-api-infographic

FREE MOBILE CLOUD COMPUTING CONCEPTS - TRAINING_MODULES_WITH_TONS_OF_VIDEOS

API, an abbreviation of application program interface, is a set of routines, protocols, and tools for building software applications. A good API makes it easier to develop a program by providing all the building blocks. A programmer then puts the blocks together.

Most operating environments, such as MS-Windows, provide an API so that programmers can write applications consistent with the operating environment. Although APIs are designed for programmers, they are ultimately good for users because they guarantee that all programs using a common API will have similar interfaces.

This makes it easier for users to learn new programs.

cloud-api-comparison-cloud-databases

APIs are critical to cloud computing services, regardless of whether they're public, private, or hybrid. However, most developers don't consider how their APIs should work; as a consequence, many otherwise solid clouds don't provide good programmatic access.

This applies to those creating private, community, and hybrid clouds for the enterprise, as well as full-blown public cloud providers.

All clouds and cloud APIs are very different, and the lack of standards and common approaches has led to confusion around the use of APIs. The results are unproductive cloud deployments and APIs that are changing faster to correct bad past designs than cloud managers can keep up with.

API design should focus on purpose and simplicity. Damian Conway offers good advice on cloud API design:

1. Do one thing really well.
2. Design by coding.
3. Evolve by subtraction.
4. Declarative trumps imperative.
5. Preserve the metadata.
6. Leverage the familiar.
7. The best code is no code at all.

I boil all of this down into three key approaches.

First, simplicity leads the day. Many APIs are built to do everything, but with such high demands placed on them, the APIs become much less useful for practical applications. My simple rule: When in doubt, break them out. Consider a finer-grained approach.

Second, consider performance. Often created as an afterthought, poorly performing APIs seem to be an epidemic.

Make sure to code as efficiently as you can and test, test, test.

Finally, design holistically. APIs have to work and play well together, so they need common data structures and usage patterns. APIs support systems -- they are not systems unto themselves. They need to adhere to common design patterns and supporting infrastructure, including governance, security, and data.

cloud-api-standards-evolution

/cloud-api-key-sign-in

cloud-api-netflix

cloud-api-markerting-three-ways

cloud-api-netflix

The data behind all those APIs, of course, is sitting in databases of one kind or another. Can we imagine browsing those databases in the same standard way we browse websites? Tim Berners-Lee did. In the original proposal for the World Wide Web, back in 1990, he listed a number of “future paths” including:

A server automatically providing a hypertext view of a (for example Oracle) database, from a description of the database and a description (for example in SQL) of the view required.

That future is real today in some places. At Netflix.com, for example, you can browse the catalog as a switch-hitter.

Starting with Genres, for example, you can proceed to B-Horror Movies, one of which is “Curse of the Voodoo”, and then proceed to “Curse of the Voodoo”‘s directors, the only one of which is Lindsay Shonteff. What you see when you click these links will depend on how your browser displays documents written in the Atom feed format. By default most browsers display such documents in a friendly way. When you turn off that friendly display, you’ll see the raw data document instead.

If you drill all the way down to an individual item of data, though, you’ll get the same result either way. Lindsay Shonteff’s name, for example, has “its own homepage” in Netflix’s web of data.

In this case the mechanism for “automatically providing a hypertext view” of the Netflix catalog is the Open Data Protocol (OData).

It’s one way, and I think a very good way, to make The Web is the API a true statement. Since OData is a Microsoft invention, and I work for Microsoft, you are entitled to be skeptical. But if you’ve followed my work and interests for the past 20+ years you’ll know that OData is the sort of technology I’ve always championed: fundamental, non-proprietary, game-changing.

Consider these two “homepages” for “Curse of the Voodoo”:

Netflix: Curse of the Voodoo [ odata.netflix.com/Catalog/Titles('5vSc') ]

eBay: Curse of the Voodoo [ ebayodata.cloudapp.net/Items('190556034358') ]

The first is an item in the Netflix catalog. The second is an item on eBay. These two webs of data are navigable in the same way. So here’s the synopsis from Netflix:

Synopsis: Director Lindsay Shonteff’s campy B-movie thriller centers on big-game hunter Mike Stacey (Bryant Haliday), who finds himself on the wrong side of a tribe of vengeful lion worshippers after he kills a sacred lion during an African hunting expedition. When the effects of the tribe’s curse begin to surface after his return to his London home, Mike realizes that his only recourse is to return to Africa and confront the witch doctor who hexed him.

[ odata.netflix.com/Catalog/Titles('5vSc')/Synopsis/$value ]

And here’s the price of the eBay item:

Price: 1.95.

[ ebayodata.cloudapp.net/Items('190556034358')/CurrentPrice/$value ]

In order to correlate these two pieces of information I’ve had to learn nothing about the Netflix or eBay APIs.

I have simply navigated two webs of data, and done so in a uniform way. Whether by means of OData or some other mechanism, I’ll want the webs of data that constitute my personal cloud to work the same way.

cloud-api-management-tag-cloud

cloud-api-infrastructure

The flexibility and economic benefits of cloud computing have generated a tremendous amount of interest. As developers work with this technology, an obvious concern is vendor lock-in. Writing an application that makes the most of cloud computing is great. But what if that application locks you in to a single vendor? The Simple Cloud API is an effort by multiple cloud vendors to create a single API that works with cloud services from multiple providers (see Resources). This article is a high-level overview of the API and its goals.

The state of the art

Cloud computing promises to change IT as dramatically as anything since the rise of the Web. If you're a developer or architect, you should at least be evaluating cloud computing if you're not using it already. These are exciting times, but the usual caveats apply. As with any new technology, you should understand cloud computing's strengths and weaknesses and use it only where it's appropriate.

One of the biggest risks with any new technology is building critical applications that lock you in to a particular vendor. Cloud computing can have significant economic benefits, but if using the cloud puts you at the mercy of a vendor's pricing strategy, that's a major concern. Even if you're perfectly happy with your cloud provider, your partners, customers and suppliers will likely use different cloud providers. It's vital that your applications be able to work with as many providers and services as possible.

The Simple Cloud API is designed to provide a single, simple, interoperable API for multiple cloud services and multiple cloud providers.

Levels of APIs

There are several ways of writing code to work with cloud services. To put the Simple Cloud API in context, we'll look at APIs at four levels:

• The wire
• Language-specific toolkits
• Service-specific toolkits
• Service-neutral toolkits

We'll consider these in detail next.

The wire

At this level, you are concerned with the actual bytes that go across the connection from the application to the cloud. For a SOAP service, the application has to build the <SOAP:Envelope> with the appropriate contents and headers. For a REST service, the application has to create the correct HTTP headers and build a URL that contains the appropriate parameters. For example, a REST request to the Nirvanix Internet Media File System (IMFS) to list all the items in the directory /dougtidwell might look like Listing 1.

GET /ws/IMFS/ListFolder.ashx?sessionToken=8da051b0-a60f-4c22-a8e0-d9380edafa6f&...HTTP/1.1
         Host: services.nirvanix.com Date: Wed, 20 Oct  2009 12:00:00 GMT

This is a request for a list of all the folders in the directory /dougtidwell. The request includes authentication information (the sessionToken parameter above), along with parameters like path name, page number, and page size. The complete URL in Listing 1 would include FolderPath=/dougtidwell&PageNumber=1&PageSize=5.

Language-specific toolkits

A language-specific toolkit provides convenience classes to create the SOAP and REST data structures. As a developer, you are still focused on the data structures that pass between your application and the cloud, you don't have to create the data structures directly. For example, the Zend_Soap library contains classes to simplify invoking a SOAP service. Invoking a SOAP service looks something like Listing 2.

$params = array(..., 'FolderPath' => '/dougtidwell',
         'PageNumber' => 1, ...); $soapClient->call('listFolder', $params, $namespace);

The powerful curl libraries are a useful way to work with REST services. A request to retrieve data via REST might look like Listing 3.

$curl_proc = curl_init('http://services.nirvanix.com/ws/IMFS/ListFolder.ashx');
         $curl_post_data = array('sessionToken' => '8da051b0-a60f-4c22-a8e0...',                          'folderPath'   => '/dougtidwell',
                                 'pageNumber'   => 1,                          'pageSize'     => 5); curl_setopt($curl_proc,
         CURLOPT_RETURNTRANSFER, true); curl_setopt($curl_proc, CURLOPT_POST, true); curl_setopt($curl_proc, CURLOPT_POSTFIELDS, $curl_post_data);
         $response = curl_exec($curl_proc);

In both of these examples, the application isn't concerned with parsing XML or checking HTTP return codes or similar issues; the toolkits handle most of the details. On the other hand, the application is not focused on business objects.

Service-specific toolkits

At this level, you are focused on business objects. Using higher-level objects that encapsulate cloud services means you no longer have to think about the format of the data on the wire. Working at this level, you have no idea whether the underlying service is SOAP or REST; you simply invoke the service. Two examples of service-specific toolkits are the Zend Framework classes for Nirvanix and Amazon S3. Following is the method for listing all of the items in a Nirvanix directory.

$auth = array('username' => 'your-username',   
                     'password' => 'your-password',                'appKey'   => 'your-appkey'); $nirvanix = new Zend_Service_Nirvanix($auth);
         $imfs = $nirvanix->getService('IMFS');  $args = array('folderPath' => '/dougtidwell',                'pageNumber' =>
         1,                'pageSize'   => 5); $stuff = $imfs->ListFolder($args);

Nirvanix provides a number of services. The example here uses IMFS. Calling the getService() method returns an object you can use to interact with a particular service. Passing parameters to the ListFolder method returns an array of the items in the folder.

In contrast to Nirvanix's directory structure, Amazon's S3 uses buckets. Buckets cannot contain other buckets, so the hierarchical structure of the Nirvanix file system isn't supported. The simpler data model of S3 is reflected in the code for listing all of the items in an S3 bucket.

$s3 = new Zend_Service_Amazon_S3($accessKey, $secretKey); $stuff = $s3->getObjectsByBucket($bucketName);

This sample code retrieves the meta information about all the items in a particular bucket. The array $stuff has metadata about each of those items, such as the item's name, size, content type, and timestamp.

Service-neutral toolkits

Service-specific toolkits improve developer productivity. However, a service-specific API locks you in to a particular service and provider. The goal of the Simple Cloud API is to provide a set of high-level classes that let you write one piece of code that can work with multiple services and providers. In looking at the Nirvanix and S3 classes, listing the contents of a directory or bucket required the ListFolder() and getObjectsByBucket() methods, respectively. In the Simple Cloud API, the ListItems() method does the job for both services.

The Simple Cloud API

The Simple Cloud API is designed for interoperable code. Operations defined in the Simple Cloud API are supported by many cloud services. The ultimate goal is that the code written to work with one cloud service should work with all similar cloud services. The PHP implementation of the Simple Cloud API uses the Factory and Adapter design patterns. To work with a particular cloud service, you call the appropriate factory method (such as Zend_Cloud_Storage_Factory for file storage) with an array of configuration parameters. The class returned by the factory method is an adapter to a particular cloud service. The adapter maps the Simple Cloud API calls to the service-specific calls required by each cloud provider.

The Simple Cloud API is defined for three types of cloud services:

• File storage
• Document storage
• Simple queues

File storage refers to traditional cloud storage systems like S3 and Nirvanix. With a file storage service, you're responsible for knowing what data you have stored in the cloud. You can get a list of directories/buckets and a list of the files in each, but it's up to you to keep track of what each file represents. Typical methods in this API are fetchItem(), listItems(), deleteItem(), and fetchMetadata().

Document storage includes more-structured systems, such as Amazon's SimpleDB. Unlike simple file storage, document storage provides query facilities to help you find information. In some cases, the underlying service will be a relational database with schema support. In other cases, it will be a much simpler type of service. Typical methods in this API are listCollections(), listDocuments(), insertDocument(), and query().

Simple queues are queueing systems like Amazon's Simple Queue Service. Typical methods for the simple queue API include sendMessage(), listQueues(), and peekMessage().

A simple sample

Note: The Simple Cloud API is still a work in progress, and details are subject to change. The sample application here works with the adapters for Amazon S3 and Nirvanix IMFS as of the date of this article's publication.

To illustrate the interoperability provided by the Simple Cloud API, we'll build a simple PHP page that works with Amazon S3 and Nirvanix IMFS. The code begins by using the Zend_Config_Ini class to create an array of credentials from an .ini file. (The factory classes in the Simple Cloud API require an array of credentials as a parameter.) The two configuration files used in this sample are s3.ini and nirvanix.ini. If the name of a configuration file is not passed as a parameter, the code uses s3.ini by default.

Listing 6. Creating an array of credentials

require_once 'Zend/Cloud/Storage/Factory.php'; require_once 'Zend/Http/Client/Adapter/Socket.php'; require_once 'Zend/Config/Ini.php'; if (array_key_exists('configfile', $_POST)) $configFile = $_POST['configfile']; else $configFile = '../conf/s3.ini'; $credentials = new Zend_Config_Ini($configFile);

 

The sample configuration file for an Amazon S3 adapter looks like Listing 7.

Listing 7. Configuration file for Amazon S3

http_adapter = "Zend_Http_Client_Adapter_Socket" storage_adapter = "Zend_Cloud_Storage_Adapter_S3" bucket_name = "developerworks" aws_accesskey = "12345678901234567890" aws_secretkey = "ABcdEFghIJklMNopQRstUVwxYZ01234567890abc"

 

The storage_adapter value is the name of the PHP class that implements the StorageService interface. For S3, the class is Zend_Cloud_Storage_Adapter_S3. A Nirvanix configuration file requires an application name (currently represented by the parameter auth_accesskey) and a directory name, in addition to a username and password.

Listing 8. Configuration file for Nirvanix IMFS

storage_adapter = "Zend_Cloud_Storage_Adapter_Nirvanix" auth_accesskey = "12345678-90ab-cdef-1234-567890abcdef" auth_username = "nobody" auth_password = "xxxxxxxx" remote_directory = "/dougtidwell"

 

The configuration file has the details that the Zend_Cloud_Storage_Factory class needs to create the appropriate adapter. Following is the code to get the list of items from the cloud storage service.

Listing 9. Using the factory class

<ul style='font-size: 125%; font-family: monospace; font-weight: bold;'> <?php $stuff = Zend_Cloud_Storage_Factory::getAdapter($credentials)->listItems(); foreach ($stuff as $nextItem) { echo "<li>".$nextItem."</li>"; } ?> </ul>

 

Using the default configuration file means that the information from s3.ini is used. That displays the contents of the S3 bucket named in the configuration file.

Figure 1. Bucket or directory listing with default configuration file

The bottom of the screen has two buttons that let you reload the page with a particular configuration file. Clicking on Create adapter using nirvanix.ini runs the exact same code, but with different configuration parameters. The results display the contents of the Nirvanix directory named in the configuration file.

Figure 2. Bucket or directory listing with a different configuration file

These two sets of results were generated by the exact same line of code. The configuration file changed, but the application itself did not.

---

Summary

The Simple Cloud API is a cooperative effort of several major cloud vendors to create a single, simple, interoperable API that works with many cloud services and providers. Work is under way to add support for more cloud services and cloud providers, and implementations of the API in languages other than PHP are in progress, as well. Cloud computing won't reach its full potential without openness and flexibility. The Simple Cloud API is an important tool for keeping the cloud open and your applications flexible.

cloud-api-kit