Section 8: ECS

by Blake Meike, former teaching fellow

Efficient Unique collections

Question: Is there a better way than the following, to select unique categories from the Scamazon catalog:

//category[ not (text()=following::category/text())]

Discussion: The solution above appears to have two problems:

It looks for all categories anywhere in the document. If new category elements are added anywhere, they will suddenly appear the query result. This is not very robust.
This is approximately an O(n^2/2) algorithm. For each category element it visits every succeeding node in the document. That seems extremely wasteful.

In the past we've reduced the size of the search by using the following-sibling axis, instead of the following axis, to dramatically reduce the number of nodes searched in the constraint. It turns out that we can still use this idea, even though the values that we want to make unique are not all children of the same parent. We do it simply by backing up the tree to things that are siblings:

/items/item[not(category=preceding-sibling::item/category)]/category

This query is safer and faster.

Using Amazon ECS

Amazon ECS, is Amazon's E-commerce Web services API. You will need it in Project 4, question 11. The only real problem with it is that it isn't documented very well. I have found two pages that, in addition to ECS.java, in the project 4 distribution, may be helpful:

The first one is the documentation pages, the second some working code.

The first thing to do is to get yourself registered as a developer. Do this now. It may take some time, and it will prevent you from doing anything else.

Background

There are a couple of architectural features that you'll probably want to grok before you embark with ECS. The first is the Response Groups

Where you might expect an object oriented architecture to define a base class, containing common attributes, and subclasses that specialized the base class with appropriate additional attributes, ECS appears to define a single class with all of the attributes that any object might have. It then populates only the relevant attributes, leaving the rest null.

Response groups seem to give some control over what gets populated, and what doesn't. They are represented as case-sensitive strings. The response groups "Large" and "ItemAttributes" seem to work pretty well. You will have to experiment to find out what others do. You can find the list of ResponseGroups by starting at the documentation page and navigating to API Reference -> ResponseGroups.

Search Index values seem to be, sort of, the "floor" of the department store on which you are searching. Third floor: power tools and kitchen items. A list of Search Indices can be found, starting on the documentation page above, by navigating to API Reference -> Search Index Values. Again, these are case-sensitive strings: be sure you spell them correctly.

Some code

ECS.java, the sample servlet, works. I very, very highly recommend that you grab the essential parts of that servlet, put them in your own, small, stand-alone java application, and modify things slowly and carefully, making sure that it never stops working. ECS is very complex. If you ever find yourself with code that doesn't work, you will have a very, very hard time debugging it.

First create the connection:

new AWSECommerceServiceLocator()
	.getAWSECommerceServicePort()

Next create the query:

ItemSearchRequest req = new ItemSearchRequest();
req.setResponseGroup(RESPONSE_GROUPS);
req.setSearchIndex(SEARCH_INDEX);
req.setKeywords(item);

_ItemSearch search = new _ItemSearch();
search.setSubscriptionId(SUBSCRIPTION_ID);
search.setRequest(new ItemSearchRequest[]{req});

This query will search for keywords matching the value of the variable "item".

Next run the query:

for (int i = 0; i < RETRIES; i++) {
	try { return connection.itemSearch(search); }
	catch (Exception e) {
		if (err == null) { err = e; }
		sLogger.warn("Failed connecting to Amazon", e);
	}
}

It seems to be a good idea to retry once or twice: connections do seem to fail every now and then.

Finally check for errors and format results:

_Items[] items = resp.getItems();
_Errors errors = items[0].getRequest().getErrors();
if (errors == null) {
	formatResponse(items);
	return;
}

_Errors_Error[] e = errors.getError();
System.out.println("Error:");
for (int i = 0; i < e.length; i++) {
	System.err.println("    " + e[i].getMessage());
}

While I can't guarantee it, it appears that all items contain references to the same error, thus items[0].getRequest().getErrors() will get whatever error there is.

Hints and trick

The most important thing it to start with working code, and keep it working.
As mentioned above, you will have to get an ECS subscription id, in order to use the service. This id identifies you to ECS: it is your account. It is probably best not to hard code it into your program. The following snippit suggests a way to do that:
```
public static final String SUBSCRIPTION_ID = System.getProperty("amazon.ecs.id");
```
If, when you start tomcat, then, you simply add:
```
-Damazon.ecs.id=098109987233408977414234
```
to the startup command, you never need to include the id in your code.
The example search uses keywords. You can search on other features, instead. Look at the definition for com.amazon.soap.AWSECommerceService.ItemSearchRequest in your "src" file to see what fields there are. It appears that if you search on more than one, the fields are 'anded' together (that is, you will get only results that meet all of the constraints you set. To search on book author you would say:
```
ItemSearchRequest req = new ItemSearchRequest();
req.setResponseGroup(RESPONSE_GROUPS);
req.setSearchIndex(SEARCH_INDEX);
req.setAuthor(item);

...
```
Be careful using a functional programming style: many of the fields in the result that ECS returns may be null. Code like:
```
item[j].getItemAttributes().getListPrice().getFormattedPrice()
```
is sure to generate NullPointerExceptions.
Sorting does something, but maybe not what you'd expect. To sort add a sort specifier to your search:
```
ItemSearchRequest req = new ItemSearchRequest();
req.setResponseGroup(RESPONSE_GROUPS);
req.setSearchIndex(SEARCH_INDEX);
req.setAuthor(item);
req.setSort("pricerank");

...
```
again, the sort specifier is a case-sensitive string. The various legal values can be found from the documentation page above, at API Reference -> Sort Values

Go wild