I am currently working on an iPhone application that is pulling data from a lot of different websites and bringing them together. It seems a little tedious to parse each website and deal with changes. It is especially difficult to do with website that are changing, and do not have an API you can use instead.
To overcome the lack of uniformity in these websites I used the Mozenda web data extraction system. It does help that I work for them so I do get a free account, but it is easy to gather the data from all the different websites using their tools. They also have a generic API which makes it easy to get all your data with one simple call. Anyway, thats not the point of my post. I want to show how easy is is to access data from the Mozenda API on the iPhone.
To be able to pull data from the API you need to have your Mozenda web service key which and the ID of the view of which you wan to pull data out of. The data comes back as xml and is in a generic format for every view. I’ve created some simple classes in Cocoa to do the xml parsing , and you are free to use it whenever you want. I have been playing with it to pull down data on prices for items at different stores. It made it easy for me to compare prices on my iPhone while I was shopping.
The code is split into three different classes. The class that does all the parsing is the MozendaParser. To use this class you must first set your Mozenda API Key in the MozendaParser.h file, and then make a call to getData with your viewID. The class will also let you do dynamic calls by passing in view parameters. Look at MozendaParser.h for more details
MozendaParser.h
#import "MozendaItem.h"
#import "MozendaView.h"
#define API_KEY @"YOUR API KEY GOES HERE"
#define BASEURL @"http://api.mozenda.com/?Service=Mozenda10&WebServiceKey=%@&Operation=View.GetItems&ViewID=%d&PageNumber=%d"
#define PARAMURL @"%@&ViewParameter.%@=%@"
// Mozenda Parser
// The parser is used to pull data from the mozenda api
// To use the parser set your api key in the API_KEY constant above and call getDataForView
// passing in a view id to get back a list of MozendaItems
// These items will correlate to the view you have set up with the mozenda system
@interface MozendaParser : NSObject {
NSMutableArray *list;
NSMutableString *currentElementValue;
MozendaItem *currentItem;
MozendaView *view;
int parseState;
}
@property (nonatomic, retain) MozendaView *view;
- (MozendaParser *) initParser;
- (void) handleParseStateRootEnd:(NSString *) elementName with:(NSString *) elementValue;
- (void) handleParseStateViewEnd:(NSString *) elementName with:(NSString *) elementValue;
- (void) handleParseStateListEnd:(NSString *) elementName with:(NSString *) elementValue;
- (void) handleParseStateItemEnd:(NSString *) elementName with:(NSString *) elementValue;
+ (MozendaView *) parseAddress:(NSString *) urlString;
+ (MozendaView *) getDataForView:(int) viewID;
+ (MozendaView *) getDataForView:(int) viewID onPage:(int) pageNum;
+ (MozendaView *) getDataForView:(int) viewID usingParams:(NSDictionary *)params;
+ (MozendaView *) getDataForView:(int) viewID usingParams:(NSDictionary *)params onPage:(int) pageNum;
@end
MozendaParser will go through the XML and create a MozendaView which contains meta data about the view, and the data. The MozendaView class has the meta data on what page you have requested, and how many items are on that page.
MozendaView.h
@interface MozendaView : NSObject {
NSMutableArray *Items;
int PageNumber;
int TotalPages;
int ItemsOnPage;
}
@property int ItemsOnPage;
@property int TotalPages;
@property int PageNumber;
@property (nonatomic, retain) NSMutableArray *Items;
- (MozendaView *) init;
@end
Each row or item returned from the API call is in a MozendaItem which is basically an NSDictionary. The keys are the columns of your view. I’ve wrapped the dictionary and offered some basic accessor methods. I almost used the NSDictionary directly, but decided to make it slightly uniform. You can pull it out if you want to.
MozendaItem.h
@interface MozendaItem : NSObject {
NSMutableDictionary *data;
}
@property (nonatomic, retain) NSMutableDictionary *data;
- (NSString *)getForKey:(NSString *)key;
- (void) addData:(NSString *) item forKey:(NSString *) key;
- (MozendaItem *)init;
@end
As you can see it is very simple, but it does allow access to the Mozenda API. Currently it does not have any error handling and only supports view support. I do plan on extending the classes and making it a full library to support Mozenda API calls in Cocoa.
You can download the Mozenda API for the iPhone.
I have done some testing with this, but not extensive. If you see any major flaws please leave a comment so I can fix it, and put and update out. I would like this available so people can use this great service on their iPhone apps.
What do you think? Is this helpful at all? Any ideas on what I should change?
#1 by stefan andreasen at March 10th, 2009
There is actually a free (non-commercial) way to scrape the web, http://www.openkapow.com
The REST services on openkapow can easily be imported to any mobile application development environment using the WADL descriptor.
The guys from Kapow and IBM made a really cool iPhone app in just 3 hours. It is integrating realtime with your personalized agenda on the coming Web 2.0 Expo website.
You can check the finished iPhone application here:
http://www.eglplanner.com/web20expo/
If you plan on going to Web2.0 Expo, then this will become you invaluable friend at the show.