2. Architecture

LoaMA is a personal media aggregator capable of handling text, audio and video media (from simple audio/video streaming to phone calls), surfing the net and collecting RSS feeds.

LoaMA provides mechanisms that allow each user to customize the way content is manipulated and routed, for instance filtering before reading/playing it. Filtering may achieved by composing several transformation steps (that we call “Transformer”). Each transformer performs a different action on its input, like filtering, merging, splitting and others as explained later.

Suppose for instance that the user is interested in finding among his collections, all audio clips by a single author. This task can be achieved composing three transformers: the first one gathers the contents from a location on disk creating a feed consisting of metadata, the second one selects the audio files and the third one filters the results searching for the given author among the collected feed metadata.

A fundamental design decision in LoaMA is to handle all data uniformly as RSS feeds: a single data type for RSS feed is used to represent both material retrieved from external sources and produced by the system and used internally by the system (e.g. playlists for visualization). This approach provides both simplicity and flexibility.

The development of LoaMA has been carried out by a team of about 70 students split into several groups. We will briefly sketch the following LoaMA modules:

·       Retriever

·       Transformer

·       Data Flow

·       Server

·       User Interface

·       Mobile

2.1. Retriever

A retriever object is created for handling the retrieval of a given URL. The gathered content is stored in a specified location on disk. If the requested document is not an RSS feed, the retriever also creates a simple RSS feed file, consisting in a single item, containing metadata about the document and an enclosure referring to the actual local copy. This allows the rest of the system to operate uniformly just on RSS feeds.

Retrievers use methods from the FeedSubscriber interface that their callers must provide, in order to notify them about the progress and completion of a download.

Various kinds of retrievers are available, each one handling a specific protocol. The currently supported protocols are:

·       http

·       ftp

·       BitTorrent

·       Skype

The Skype interface provides the capability to issue VoIP phone calls while the other protocols allow file downloads from the Internet providing the typical features of a media aggregator.

The retriever module dispatches network requests to the appropriate retriever and keeps track of the status and progress of each download.

The search module, which allows searching through the collected contents as well as on local disk, also produces its results as an RSS feed.

A cache mechanism is available in order to reduce latency of retrieval.

2.2. Transformers

Transformers are used in LoaMA for most of the internal system tasks and are exposed to users who can combine them in any way to achieve their needs. Transformers operate on RSS feeds of any kind and produce RSS feeds.

Composability of transformers is achieved by defining a single interface that each transformer must implement:

interface ITransformer {

  RssFeed[] Input { get, set }

  RssFeed  Output { get }

}

i.e. a transformer accepts an array of RssFeed as input and produces a single RssFeed as output. There is no lack of generality here since a RssFeed object may contain several channels, which may possibly be split into separate feeds.

A number of simple transformers have been built and are currently available:

·       Merge: merges several feeds into one

·       Filter: select items from a feed according to a certain criterion, for instance selecting feeds from a specified source or in a specified format

·       Split: splits one feed into many

·       Shuffle: rearranges the items in a random order

·       Sort: sorts the items according to a certain criterion.

The last two transformers are also used internally by LoaMA to operate on playlists, exploiting the fact that playlists are themselves represented as RSS feeds.

Some transformers accept configuration options to control their operation: for example, a filter transformer must be given the filtering criteria to be applied to an RSS feed.

A few more complex transformers are also available:

·       Indexer: creates a searchable index for a collection of feeds

·       Classifier: divides the items in a feed into a set of predefined classes

·       Clusterer: groups channel items into sets with similar content.

The Indexer transformer is helpful in providing faster search through user collections. The indexer creates a searchable full-text index by analyzing the text documents in the user collections (HTML, RSS documents), or by extracting metadata (e.g. author, title, etc.) from other media types. Such metadata are extracted and stored in the RSS feed document created by the retrievers.

2.3. Data Flow

The Data Flow module bestows the possibility of customizing LoaMA. A LoaMA user can compose transformers to perform tasks such as classifying, collecting and filtering contents. The user can automate the process of creating his own playlists, to organize and classify his material, and to share it with other LoaMA users.

Figure 1. A Composition of transformers.

Figure 1 illustrates a data flow where a merger transformer merges three retrieved feeds, two other feeds are filtered and the two output feeds are classified to produce the final feed, that will consists of several channels, one for each classification category.

Transformers can be built through the LoaMA user interface: the user selects which feeds he wants to transform and then selects from a window pop up among a number of available transformers to apply to them. The resulting output feed of the transformation is added to a list of temporary feeds, appearing in a specific pane in the user interface (bottom left in Figure 1), so that it can be selected again as input to further transformations. The composition of transformers creates a data flow of RSS feeds, which is itself a transformer of type CompositeTransformer that can be saved for use in creating other compositions. This step involves generating code for the composite transformer and compiling it into a new assembly (DLL) that becomes part of the available transformers loaded at application start up.

Figure 2. A Composite transformer.

A Data Flow is made out of TransformerNode’s connected to each other. The Data Flow process is demand driven: the resulting RSS feed of a transformation is obtained by calling the Start() method on a TransformerNode. The TransformerNode in turn invokes Start() on all of its input transformers. This triggers a cascade of requests on each component of the dataflow, until the request reaches a retriever or a primitive transformer with no inputs. A TransformerNode is notified through its FeedSubscriber interface when a feed becomes available for processing. When the transformer’s input feeds are all available, it applies its transformation on the collected feeds in order to produce its output feed and notifies its caller.

Data Flow transformations occur between a local RSS feed retriever and a user interface Presenter. A dummy TransformerNode is used to interface transformers to locally stored RSS feeds.

Applying different transformers to his feeds allows a LoaMA user to customize every step in handling feeds, from retrieval to selection, presentation, storage and distribution.

2.4. Server

This module provides a server facility to LoaMA, so that users can publish the material they wish to make available to others either through the Web or other file sharing protocols.

The server currently implements a subset of the HTTP protocol, including GET requests and HTTP authentication to restrict access. The server also supports the BitTorrent protocol, implementing the operations for startup, shutdown and logging.

The server module consists in:

• Web Server
• Throttler
• Access Control
• Logger
• BitTorrent
• Monitoring
• Streaming

The Web Server handles all incoming requests from external users, either general Http or BitTorrent requests. In the latter case, the BitTorrent module is invoked.

The throttler has the task to control the overall use of bandwidth within LoaMA. This is a difficult task, since various components of LoaMA have different requirements and employ protocols that do not incorporate provisions for bandwidth control. Bandwidth control is essential in a media aggregator which can consume bandwidth eagerly, both in download and upload.

Several alternatives where considered in the design of the throttler: a drastic solution would have been to force all retrievers/uploaders to use a special socket interface incorporating bandwidth control. This would have been impractical because it would have required re-implementing all protocols from scratch. A simpler solution was chosen where the throttler assigns an upper amount of bandwidth to each of the concurrent retrieval threads according to a user assigned priority. Each thread is requested to comply within this limit. This restrain mechanism has been implemented in the http retrievers, which exploit IO completion ports, and hence can be delayed when the overall bandwidth limits is achieved. BitTorrent 4.0 already provides a bandwidth limitation mechanism that the throttler can exploit.

The BitTorrent module serves requests for files having the .bt extension (requests for metainfo .torrent files are handled instead by the Web server): it runs a tracker and implements the BitTorrent peer wire protocol responding to requests from BitTorrent clients for a piece of a file.

The Access Control module implements a login and password mechanism to manage the authentication for users that want to access to the server.

The logger records the user actions, since the startup of the application and it writes error messages to a log file.

The Monitoring module displays information about the current uploads from other network users. The module allows controlling bandwidth usage, for instance terminating some very demanding transfers.

The Streaming module sends multimedia files, such as mp3, wma or other, through http streaming. It displays information regarding the streaming process, e.g. the size and name of the file being transferred, the progress and status of the transfer. It is also possible to kill one or all of the existing uploads.

2.5. User Interface

The UI module provides an intuitive, graphical interface for reading the collected media and for controlling the facilities of the LoaMA application.

Figure 3. The LoaMA user interface.

Main Window

The main window of LoaMA handles the typical interaction controls (toolbar, menu bar) and it is split into several panes, each one handled by a specific presenter.

Playlist

The Playlist Presenter displays the tree structure of the current playlist.

A playlist is organized in categories and the user can manipulate the folders corresponding to each category.

An interesting feature of LoaMA is that playlists are themselves represented as RSS feeds, so that they can be exported and exchanged with other users. For the same reason playlists can be manipulated and transformed by any of the LoaMA transformers: for instance they can be merged, clustered or classified, merged with annotations, analyzed by statistical tools, or passed to recommendation systems to obtain suggestions for other potentially interesting items to the user.

Keeping track of read items in a playlists is also dealt by manipulating RSS feeds: an RSS feed is generated which contains additional metadata about the items in the playlist, e.g. whether the item has been read, where it is stored locally and so on. Whenever the playlist is loaded, it is merged with such additional metadata feed to present the user with its personal view of the playlist.

RSS Presenter

The RSS Presenter displays, for each RSS feed, the list of items present in its channel.

When an item is selected from this list, its content is transformed from XML into a stylized HTML and shown in the Web Browser tab below.

This module interacts with other UI modules: for instance after selecting an item, the appropriate player can be started.

Multimedia Presenter

The Multimedia Presenter gets invoked to start the presenter suitable for a specific media: the browser for HTML documents or the media player for audio/video files.

Download Manager

This module creates the appropriate retriever to perform a requested download for a URL and returns an observer object, which can be used to monitor the progress of the download, to suspend/resume or stop it and also to throttle the amount of bandwidth it uses.

2.6. Mobile

A version of LoaMA has been developed for cellular phones. This module has been developed on the Java platform, because of its availability on a wide range of mobile phone models.

Mobile LoaMA provides the following features:

·       RSS 2.0 support: implementation of a RSS parser and different RSS viewer for all possible content types of RSS feed entries.

·       Embedded Media Player: a player for multimedia content (audio and video) of user playlist implemented with MMAPI.

·       Embedded Image Viewer: a viewer for user photo gallery or playlist.

·       Embedded HTML Browser: a browser to visualize web contents.

Figure 4. LoaMA on a cell phone.

The application also supports optimizations for RSS feed download (e.g. caching of downloaded feed, proxy) and search (RSS filters).