You might already have read what Aggreg8 is but if you happened to have stumbled across this document on your web travels , I will give you a brief intro to what Aggreg8 is.

Right then now that I am sure you know what Aggreg8 is I can begin this technical manual. To the right you can see the Table of Contents. You can navigate to any part of the Technical manual using this table.

1.₁ What is the purpose of this document?

This took me a while to figure out so here goes: The primary purpose of this document is for any developer to be able to pick up this manual and by the time thay have finished reading they will have a very good idea of how Aggreg8 was built and what technolgies were used to build it, and they will (I hope) have enough knowledge to be able to change features or add features to Aggreg8 with ease. They will know all about Mozilla and the technologies that make Mozilla such a great framework to build applications on. The reason I see this as a main goal of the manual is that I have released this project (informally) to the Open Source community (mainly MAD [Mozilla Application Developers] on Mozdev). And it is for these reasons that I am going to go into some details about the architecture of Aggreg8 and the whole Mozilla XPFE (Mozilla framework).

1.₂ Introduction

Aggreg8 was a fun project to build, It is all built with brand new (to me anyway) technologies, all of which had to be researched in depth before any coding could even begin. Mozilla is a browser that is absolutely splendid, since discovering it, I rarely use MSIE, very rarely, and it was amazing to find that there are whole communities of like minded people furthering the growth of Mozilla and all it's children browsers (Firebird etc.) The Mozilla application developers (MAD) on Mozdev.org of which I am now a member (see aggreg8.mozdev.org) have huge list of add-ons that they have developed to enhance the Mozilla experience. It gave me a will to learn, one which I hadn't felt since I found PHP. As Aggreg8 is part of a final year project , I decided that just a finised product was not enough , it therefore that I created and built the Aggreg8 website (http://www.Aggreg8.net). Aggreg8.net is a kind of portal to the Aggreg8 project. It hosts all documentation, downloads and screenshots etc. but also has an Aggreg8 weblog and RSS feed so that all interested can keep up to date with progress of the Aggreg8 project. The whole site was created using PHP and a MySQL database backend. I used the weblog as a kind of project journal to keep track of project goals etc. It was very useful. Now that the Aggreg8 is complete (as far as my Final year project is concerned), version 0.0.1 has been released and has a small user base already, I can only sit back and wait and see what happens with it after my finals.

1.₃ Why Mozilla?

Right then I've gone on and on about Mozilla but what is it and why is better than MSIE (MicroSoft Internet Explorer). Mozilla is an open-source web browser, designed for standards compliance, performance and portability. Below are a few paragraphs that will try and explain, why the whole frenzy around Mozilla.

1._3.1 From the user's point of view.

Mozilla is designed from the ground up to support open internet standards across a variety of platforms including Windows, Linux, Mac OS X, OS/2, Solaris, and many more which means that if you are a developer like myself that likes to develop on several platforms , you always have the same browser installed on all of your different platforms. Mozilla provides users with acclaimed browsing convenience along with power features such as pop-up blocking and tabbed browsing. Mozilla comes bundled with a fully featured mail client, a HTML Editor, and an IRC client. With all these features plus many more, it far supersedes MSIE in features and usability. Mozilla also supports themes which means that you can download a new look for your Mozilla browser. I have found that all of the features in Mozilla make browsing a much more pleasant experience and my taskbar is definitley less cluttered thanks to tabbed browsing.

1._3.2 From the web developer's point of view.

Mozilla supports Web Standards, so you can do away with annoying browser-specific code. Mozilla is also a web development platform with advanced tools like the DOM Inspector, the JavaScript Debugger, and support for XML, DHTML, XUL, SOAP, XSLT and more. In short if your code looks and works on Mozilla then it will on any browser that claim they support web stabdards. I now develop all of my websites using Mozilla as a test a debug platform. And as mentioned before, Mozilla is multi platform so if you develop for one platform , you have devloped for many without any chages in your code. Mozilla is an Evangilists dream (Evangelism: Making sure websites and applications generate standards-compliant code.).

1._3.3 From the browsers embedder's point of view.

Mozilla uses the Gecko layout Engine to render all of its's code. The Mozilla children browsers that I spoke of before (listed below) all use this Gecko layout engine and Mozilla Components. Gecko features amazing support for open web standards and existing web content, is built from the ground up with security in mind, and is available on more than a dozen platforms - it is the clear choice for internet enabled applications. There is a very good article on O'Reilly Net entitled " Let One Hundred Browsers Bloom" discussing Mozilla, Gecko and their children browsers.

• Netscape 6 + 7
• Firebird
• K-Meleon
• Galeon
• SkipStone
• Camino
• AphroDite
• Netscape

1._3.4 From the application developer's point of view.

With tools such as those mentioned above and Mozilla's XUL (Mozilla's XML User Interface language), XPCOM (Cross Platform Component object model), XPConnect and XPInstall (Cross platform installation) technologies that make up the Mozilla framework, it is even easier to develop cross platform web-based or non-web-based applications. There is built-in support for XMLHttpRequest, XML-RPC, SOAP, and WSDL to enable Mozilla-based application authors to take advantage of the exploding world of web services. Aggreg8 is built on this Mozilla framework, without this framework , Aggreg8 would have been developed in java, C# or some such, which are not half as elegant and not at all as easy to port (especially C#). In the future when I am faced with a decision to make as to what framework to build an application, Mozilla will be my first choice.

I will explain each of the technologies above in more detail in the Architecture section of the manual. When descussing these technologies, I will be specifically discussing them in the context of Aggreg8, as well as a brief intorduction to the technology itself (I feel this is needed as they are all new technologies).

I've discussed Mozilla and why it is so great but I have yet to discuss the most important reason for Aggreg8, weblogs and RSS feeds, without these there would be no reason for Aggreg8. I have been interested in weblogs and RSS for a good while now (I started my weblog in january '02). I have written a paper on weblogs entitled "Weblogs What, How and Why?." which gives a more detailed guide to what weblogs are and why they are so popular.

2.₁ What are weblogs?

In it's simplest of simplest of forms, a weblog is a dated entry on a website. I have kept a personal weblog (http://www.philroche.net) since January '02 , You might ask , what do you write about?.. the answer is anything at all that pops into my head. Most personal weblogs are like this, all you have to do is look at weblogs.com and Blogger.com too see the amount of personal weblogs out there. There are other types of weblogs too, Project weblogs (just like the weblog I kept for Aggreg8) which are used to discuss a particular project. There are group weblogs and many other types... the list is endless. There are huge communities built around weblogs (sometimes referred to blog communities or RSS communities). People all around the world are writing weblogs about all topics from recipes to Pasta dishes to Quantum Physics. This world of weblogs is exploding faster than people can comment on it, let alone document it. Articles in many magazines and newspapers have discussed weblogs. Google, probably the most powerful and well known internet company recently bought Blogger.com (Blogger.com is a web-based weblog client), this shows that even a company as big as Google sees a future in Blogging (the terms weblog and blog mean the same thing). People all over the world read weblogs from all around the world and I think it was this fact that led me to become so interested in weblogs, that anybody can have a voice no matter how faint or strong.. somebody will listen. (that's my rant over with).

2.₂ What is RSS?

What is RSS and how does it relate to weblogs?. RSS is an acronym for Really Simple Syndication or RDF Site Summary (both mean the same thing). What this means is that, as RSS is a lightweight XML format designed for sharing headlines and other Web content or metadata about a website, RSS is used to syndicate headlines or metadata from a website. These headlines I speak of could be weblogs, most recent CVS checkins, news bullitins etc. It is just a standard way of storing all these different types of data. As an example I will describe how the RSS from the Aggreg8 weblog is structured see fig. A, at the top level you have the XML namespace and and rss namespace. In this example I am using RSS 1.0 (there are several verions of RSS, the newest is 2.0 but they all esentially do the same thing). You can see that there is a title etc, this is referred to as metadata about the feed, further down you can see the "<item>" element (circled in red in fig. A), this is where the content of each weblog or news item is stored, and it is these items that are of interest to Aggreg8. When a site publishes an RSS feed Aggreg8 is able to grab the items within the RSS feed and display them.

2.₃ Aggregators?

As there are so many weblogs and RSS feeds out there, it is nearly impossible to keep track of them all, therefore a number of rss aggregators have been developed. Some of these include Newzcrawler, Radio Userland, NewMonster, ApmhetaDesk , Aggie and of course Aggreg8. These aggregators allow the user to arrange, categorise and store their favourite RSS feeds, these feeds can then be viewed and archived, Some of the more advanced aggregators allow for downloading of rss feeds at specific intervals throughout the day so your feed archive is always up to date. Of the aggregators listed above, none work solely on Mozilla (newsmonster has been integrated into Mozilla but uses Java Webstart for the bulk of the downloading and parsing work), apart from of course Aggreg8 which is built solely using Mozilla technologies.

I have now explained what weblogs are, what RSS is, what Mozila is and why it is so great. You should also by now have a feel for what it is Aggreg8 does. I will from this point on be discussing how I built Aggreg8.
When I began developing Aggreg8 I broke the development process down into two phases.

• Phase 1: was the creation of an online RSS parser. This was completed and was included in the beta versions of Aggreg8, when the built in parser was not built. The reason for this was to become familiar with the structure of RSS and what elements I would need to parse out.
• Phase 2: was the building of Aggreg8 using all the Mozilla tools. This involved creating the GUI first then fleshing out the features. At first I used the online parser to parse hard coded in RSS feeds but later on I was able to move away from this model with dynamic RSS feeds and a client side Parser ... more on this later.

3.₁ PHP Parser?

As mentioned before the Online parser was created so that I could understand the structure of RSS and how I would go about parsing out the important details. The online parser is fully functional and is live @ http://parser.Aggreg8.net (The PHP parser code can also be viewed here). This parser was used in some of the alpha/beta versions of Aggreg8 when the client-side parser was not built yet. It is a usefull online tool if you want to read a weblog and all you have is a link to the rss feed.

I have dicussed Mozilla, but now I will explain how applications can be built on the Mozilla framework. There are several components that make up the framework (each listed below), I will discuss these components and how they were used in the building of Aggreg8. The main source of the following information is the O'Reilly Book "Creating Application with Mozilla".

4.₁ XPFE.

XPFE (Cross-platform front end). The part of Mozilla that implements the user interface of mozilla. Everything that makes a browser a browser except the rendering of pages is considered XPFE (bookmarks, toolbars, menus etc). The XPFE uses a number of existing web standards, such as Cascading Style Sheets, JavaScript, and XML (the XML component is a new language called XUL, the XML-based User-interface Language). In its most simple form, XPFE can be thought of as the union of these technologies (see fig. B). To understand how XPFE works, we can look at how these different components fit together. JavaScript (ECMAScript) creates the functionality for a Mozilla-based application, Cascading Style Sheets format the look and feel, and XUL creates the application's structure. Instead of using platform-specific code in languages like C or C++ to create an application, XPFE uses well-understood web standards that are platform-independent by design. Because the framework of XPFE is platform-independent, so are the applications created with it. XPFE can be compared to DHTML (Dynamic HTML) in that, DHTML is a combination of HTML, JavaScript, and CSS that allows a developer to create an application that is contained within the content area of a browser. XPFE provides a logical evolution to this idea by allowing the creation of applications that are more powerful, more flexible, and that can live outside the browser window as standalone programs. Aggreg8 is built on the XPFE and can be run inside the browser or outside the browser as a standalone application.

4.₂ Chrome.

Before I go into more detail on the components that make up the XPFE, I will explain the structure of a Mozilla based application. Mozilla applications live in a lovely place called "chrome". What this means is that all installed Mozilla applications can be referenced by the "chrome://" url. Chrome is a subdirectory of the main Mozilla application directory. All of the files that make up an application are located in this chrome directory in a subdirectory with a name the same as the name of the application. In the case of Aggreg8 there is a subdirectory in chrome called aggreg8 and it is in this directory that all the code for Aggreg8 resides. This application directory is further broken down as illustrated in fig. C. There are three subdirectories; Content, Locale and Skin. An application that has a file structure like this is known as a packaged application.

"Separation of presentation from logic makes applications resilient to change." -- Mozilla.org

• content : The contents directory contains all the script files used to provide functionaltiy (these would be javascript file) and GUI files used to create the user interface (these would be XUL files). Other files such as Datastore files (RDF) and html pages could also be stored here.
• locale : The locale directory is where all langauge specific files are stored. You can see in fig. C that there is a subdirectory of the locale directory named en-US. This directory contains all the US english language files. Language files as I put it can take several forms, they couldmbe html pages that are used in the application. They could also be DTD files (DTD is an XML Document Type Definition) which allow for strings to be included in XUL files, without hard-coding them in. There can also be String Bundles , these are used in javascript so that strings do not have to be hard-coded in. These String bundles work in the same way as bundle files in Java and JSP. the beauty of having your application arranged like this is that to localize the application into another language , all you have to do is edit the files in the locale directory and it's subdirectories. Aggreg8's files are all arranged in this way, so to localise Aggreg8 only 2 files need to be changed.. the DTD and the String bundle file.
• skin : As you can guess, the skin directory is used to store all the style specific files. This means that to change the appearance, colors etc, is much easier as the only files you have to cjange are those in the skin directory. All the CSS files I use in Aggreg8 are stored in the skin directory and as such changing the style of a page is made much easier than if the style were coded into the page itself.

Each of the directories: content, locale and skin all contain a file named "contents.rdf" (RDF files will be discussed later). "contents.rdf" is a package manifest, it describes the contents of the package specifically the contents of the directory where the contents.rdf file is located. For instance in the locale directory, the "contents.rdf" file would list the language that the files belong to.

Once all files are set up you must then register the application with Mozilla so that it knows that a new application has been installed. This is done by editing the "installed-chrome.txt" file in the chrome directory. Mozilla checks the "installed-chrome.txt" file when it boots up and if any changes have been made, it recognises them. Below you can see a snippet of the file. The snippet is where Aggreg8 is registered with Mozilla. Mozilla ia told that the skin files are in "resource:/chrome/aggreg8/skin/", locale files are found in "resource:/chrome/aggreg8/locale/en-US/" and content files are in "resource:/chrome/aggreg8/content/" where "resource:" is the Mozilla install directory (ie. the directory where Mozilla has been installed).

skin,install,url,resource:/chrome/aggreg8/skin/ locale,install,url,resource:/chrome/aggreg8/locale/en-US/ content,install,url,resource:/chrome/aggreg8/content/

Once Mozilla is restarted with these changes made, Mozilla will realise Aggreg8 has been installed and entering "chrome://aggreg8/content/aggreg8.xul" as a url will open the main Aggreg8 page inside the browser.

The directories in fig. C from aggreg8 down can be archived into a jar file. The directories still have the same structure only they are in an archive. This helps with keeping the chrome clean and so that users won't accidently mess up a file contents. An Archived application must be registered a little differently from the previous way. Below you can see the highlighted text which shows the changes that have to be made. The files that make up Aggreg8 are archived in this fashion and is registered with the following lines appended to the "installed-chrome.txt" file.

content,install,url,jar:resource:/chrome/aggreg8.jar!/content/ skin,install,url,jar:resource:/chrome/aggreg8.jar!/skin/ locale,install,url,jar:resource:/chrome/aggreg8.jar!/locale/en-US/

Mozilla is able to extract the files from these archives at runtime. Applications can also live outside of chrome but these would not be registered with Mozilla. Applications in chrome have more permissions than applications outside chrome, in that they can write to files etc. As Aggreg8 writes to a file to store all subscription information, it needed to be a registered application in chrome.

4.₃ RDF.

RDF (Resource Description Framework) is a way of storing data and describing that data using metadata. It can be looked upon like XML in that it has a hierarchical structure that stores and describes data. I have always thought of RDF as a datasource that describes the data that is contained within it. Mozilla uses this term datasource to describes RDF files too. Aggreg8 uses RDF to store it's subscriptions information (see fig. D). RDF has many ways of storing data, in containers such as those highlighted in green see fig. D (Aggreg8 uses RDF containers for storing the feeds in a particular category). data can also be stored as attributes of an element. Highlighted in yellow you can see that the title of the particular feed being described is "Philroche.net". This is how Aggreg8 stores the details of an RSS feed. fig. D is an example of a subscriptions file in Aggreg8, this one however has only one category ,"personal" and that category only contains one feed, "Philroche.net". Mozilla has built in features to handle queries to RDF files (will discuss these later), Hierarchical trees can be built using an RDF file as the tree's datasource. These are built in XUL using a template construct similar to the template construct in XSLT. This is how Aggreg8 populates it's Subscriptions tree in the display and subscriptions screen.

4.₄ JavaScript.

We all know what javascript is but I'll give a brief intorduction anyway. JavaScript is the Netscape-developed object scripting language, it is a superset of the ECMA-262 Edition 3 (ECMAScript) standard scripting language, with only mild differences from the published standard. Javascript is the main langauge for Mozilla Applications although there is support for Perl and Python. Aggreg8's functionality was entirely written in Javascript. There is a built in Javscript Debug console in Mozilla and once turned on, you can debug your Javascript code by using the "dump()" function. Using this function outputs to the debug console. This is a very useful tool to have, as prior to Mozilla when I wrote Javascript code, debugging was a trial and error effort. There is also a built in Javascript Debugger where you can test entire javascript scripts from the debugger (it is known as the Venkman debugger).

4.₅ DOM.

The DOM (Document Object Model) is an API for HTML and XML documents. It provides a structural representation of the document, enabling you to modify its content and visual presentation. Essentially, it connects web pages to scripts . All of the properties, methods, and events available to the web developer for manipulating and creating web pages are organized into objects (e.g., the document object that represents the document itself, the table object that represents a HTML table elements, and so forth). Those objects are accessible via Javascript. As RSS is an XML format it too has a DOM. It is using this DOM that Aggreg8 parses RSS. Just like the Javscript tools mentioned above, Mozilla has a DOM inspector built in that allow you visulaise a files DOM. Very useful when your DOM gets complicated.

4.₆ XUL.

XUL (pronounced "zool") is Mozilla's XML-based User interface Language that lets you build feature-rich cross platform applications that can run connected or disconnected from the Internet. These applications are easily customized with alternative text, graphics and layout so they can be readily branded or localized for various markets. XUL is a mark up language just like HTML and just as HTML has body and table elements etc, XUL has window and box etc.. that it uses to create it's user interface. There are many XUL elements (Element reference on XULPlanet) and it takes a while to learn what they all do and how they all behave. Each of these elements have optional attribute values that alter the behaviour of the element yet again. Once you become familiar with the base set of elements, some very usable interfaces can be built. Some of these elments are referred to as widgets , an example is the tree widget "<tree>". The tree widget is used in Aggreg8 to create the Subscriptions tree that displays all of the subscribed feeds. There are many XUL files that make up the Aggreg8 application (all of them in the content directory as mentioned before). You can see in fig. E small snippet of the XUL used in Aggreg8, this small snippet is actually the code used to place the Aggreg8 toolbar at the top of the Aggreg8 application. I mentioned before about DTD files that are stored in the locale directory. In the snippet in fig. E you can see these DTDs in action , "&menu.aggreg8;" actually corresponds to the string "Aggreg8". You can also see that XUL is quite different from other User interface languages like DHTML in that the elements are not familiar and have alot more scope (they have many different behaviours depending on the optional attribute values set). Iframes are used extensively in Aggreg8 to simulate different windows. Inside each of these Iframes is another XUL file. XUL files have a DOM and can be accessed using Javascript DOM functions. XUL can become an even more powerful user interface language with technologies such as XBL and Overlays which complement XUL inthe creation of Mozilla application user interfaces.

• The eXtensible Bindings Language (XBL) is a markup language that defines special new elements, or "bindings" for XUL widgets. With XBL, developers can define new content for XUL widgets, add additional event handlers to an XUL widget, and add new interface properties and methods. Essentially, XBL empowers developers with the ability to extend XUL by customizing existing elements and creating new elements of their own. By using XBL, developers can easily build custom user interface widgets such as progress meters, fancy pop-up menus, and even toolbars and search forms. These custom components can then be used in XUL applications by specifying the custom tag and associated attributes. Aggreg8 uses XBL in the selection trees, A listener is bound to the tree widget (see fig. F ) and once a feed in the tree is selected then an action is performed. Aggreg8 also has a listener bound to the RSS download activity, once the download is complete, an action is performed. As well as these listeners there are also broadcasters but I did not use these in Aggreg8. Listeners are also referred to as Observers.
• Overlays are XUL files used to describe extra content for the UI. They are a general mechanism for adding UI for additional components, overriding small pieces of a XUL file without having to resupply the whole UI, and reusing particular pieces of the UI. Aggreg8 used overlays like this with all of the subscriptions tree instances (see fig. F for an example of the subscriptions tree). The overlay specifies the content of the tree but the xul file would simply have an empty tree widget/element specified. I mentioned before the use of templates used to populate the tree. It was in these overlay files that this template code was used. Overlays are a powerful mechanism for customizing and extending existing applications because they work in two related but highly different ways. In one respect, Overlays are synonymous with "include" files in other languages because an application may specify that an Overlay be included in its definition. But Overlays can also be specified externally, enabling the designer to superimpose them upon an application without changing the original source. This is how the Aggreg8 option on the Tools menu of Mozilla was added.

4.₇ XPCOM + XPConnect.

The Cross Platform Component Object Model (XPCOM) is one of the main things that makes the Mozilla application environment an actual framework. It is a development environment that provides the following features for the cross-platform software developer:

• component management
• file abstraction
• object message passing
• memory management

This component object model makes virtually all of the functionality of Gecko available as a series of components, or reusable cross-platform libraries, that can be accessed from the browser or scripted from any Mozilla application. Applications that want to access the various Mozilla XPCOM libraries (networking, security, DOM, etc.) use a special layer of XPCOM called XPConnect, which reflects the library interfaces into JavaScript (or other languages). XPConnect glues the front end to the C++-based components in XPCOM. Aggreg8 uses XPCOM and XPConect to manipulate the Subscriptions RDF file. Using XPCOM and XPConnect was the most difficult part of developing Aggreg8. In short to get access to a file you need to create an instance of the Mozilla RDF file Component [1], then use XPConnect to update the file and save it back to the file using a function from the interface to that component. Below you can see some of the code from the Subscriptions datastore manipulation code. It shows how Component instances are created and an interface to the component created.

var RDFC = '@mozilla.org/rdf/container;1';^{[1]
RDFC = Components.classes[RDFC].createInstance();}
RDFC = RDFC.QueryInterface(Components.interfaces.nsIRDFContainer);

var RDFCUtils = '@mozilla.org/rdf/container-utils;1';
RDFCUtils = Components.classes[RDFCUtils].getService();
RDFCUtils = RDFCUtils.QueryInterface(Components.interfaces.nsIRDFContainerUtils);

var RDF = '@mozilla.org/rdf/rdf-service;1';
RDF = Components.classes[RDF].getService();
RDF = RDF.QueryInterface(Components.interfaces.nsIRDFService);

Together, XPCOM and XPConnect enable developers to create XUL applications that require the raw processing power of compiled languages (C/C++) or access to the underlying operating system. Without XPCOM and XPConnect, I would not have been able to manipluate the RDF file that I use to store the subscription feeds.

4.₈ XPInstall.

XPInstall (Cross Platform Install), provides a standard way of packaging XUL application components with an install script that Mozilla can download and execute. XPInstall files are XPI (pronounced Zippy) files, basically an archive of the application files and the install script. XPInstall enables users to effortlessly install new XUL applications over the Internet or from corporate intranet servers. To install a new application the user need only click a hypertext link on a web page or in an email message and accept the new package through a Mozilla install dialog. This ishow Aggreg8 can be installed (see download page and installation guide). The Aggreg8 install script downloads the aggreg8.jar archive, saves it to the chrome directory and then updates the "installed-chrome.txt" file. It is very easy for a user to install Aggreg8 using the XPInstall facility, simply click on the install Aggreg8 link and the install script does the rest. As it is cross platform , it works on all platforms.

4.₉ Putting it all together.

That's alot to take in I know, it took me a long time to understand the structure of a Mozilla application before I could even contemplate beginning to build one. Once I did have a good hold on the structure and the model of an application, I broke the development into 7 steps.

1. Building the User Interface.
   The entire user interface for Aggreg8 was built using XUL. You can see screenshots of the User interface. Grids, Trees, tabs, iframes, text boxes, buttons and many more features of XUL were used.
2. Building the subscriptions tree from an RDF datasource.
   The subscriptions trees were built inside overlays and then included inside the main XUL file using the overlay syntax "<?xul-overlay href="chrome://aggreg8/content/aggreg8overlay.xul"?>" and overlay id "<tree id="rssfeed-tree" />". Specifying the datasource as an attribute in the tree widget "datasources='resource:///chrome/aggreg8/content/rssfeeds.rdf' " inside the Overlay code and then using templates like in XSLT (see code) allowed for the data from the RDF datasource be extracted and displayed in a hierarchical tree manner. This means that the rss feeds that populate the subscriptions tree do not have to be hardcoded in to the XUL code, they can simply be added to the RDF file and the change will filter through to the subscriptions tree when refreshed.

   <template><!-- Template -->
      <rule iscontainer="true">
          <treechildren >
               <treeitem container="true" open="true" uri="rdf:*">
                  <treerow>
                      <treecell _type="category" label="rdf:http://www.philroche.net/redbrick/CA4Project/rdf#CategoryName" />
                  </treerow>
               </treeitem>
           </treechildren>    
      </rule>
      <rule isempty="true">
          <treechildren >                
           <treeitem uri="rdf:*">                
                  <treerow>
                  <treecell _type="feeds" label="rdf:http://www.philroche.net/redbrick/CA4Project/rdf#FeedTitle" url="rdf:http://www.philroche.net/redbrick/CA4Project/rdf#FeedURI" description="rdf:http://www.philroche.net/redbrick/CA4Project/rdf#FeedDescription"/>
                  <treecell label="rdf:http://www.philroche.net/redbrick/CA4Project/rdf#FeedDescription"/>
                  <treecell label="rdf:http://www.philroche.net/redbrick/CA4Project/rdf#FeedURI"/>
                  </treerow>                    </treeitem>                    </treechildren>
               </rule>           </template><!-- Template -->

3. Manipulating the Subscriptions store.
   As explanied previously, the RSS feeds that are subscribed to are all stored in an RDF file. To update this file manually everytime you found an interesting RSS feed would not be viable solution. So a method for updating the file using the XUL interface and javascript had to written. The XUL side of it was the easiest, in that all that was need was a few textboxes and a submit button. The javascript had to use XPCOM and XPConnect to query the interface to the supplied RDF datasource. This was very tricky as there is not too much documentation out there on it, most of the research was done by looking at the C++ code that makes up the XPCOM component for RDF datasources. Using the functions in the RDF datasource interface, Aggreg8 provides functionality that allows a user to Add/Edit/Delete RSS feeds and also allows a user to add categories for the feeds to be arranged in. All the code for the RDF manipulation is in the "AddRSSFeed.js" file in the content directory of the aggreg8 application. (see Code excerpt document for examples from this code)
4. Rebuilding the Subscriptions tree.
   Each time a feed is added/edited or deleted from the Subscriptions datasource then the Subscriptions tree built by the overlay must be rebuilt for the update in the datasource to be shown. This involves grabbing the in-memory version of the datasource and writing that out to the file and rebuilding the tree. Below you can see code for a javascript function "refreshSubscriptiontreefromframe" that refreshes the subcriptions tree. Simliar functions are in place that refresh all trees in Aggreg8 after an update to the datsource is performed.

   function refreshSubscriptiontreefromframe()
   {
   aggreg8dump("refreshSubscriptiontree from frame" );
   try{
       var RDF = Components.classes["@mozilla.org/rdf/rdf-service;1"] .getService(Components.interfaces.nsIRDFService);
       
       var ds = RDF.GetDataSource("resource:///chrome/aggreg8/content/rssfeeds.rdf");
       var tree = document.getElementById("rssfeed-subscriptions-tree");
       tree.database.AddDataSource(ds);
       netscape.security.PrivilegeManager.enablePrivilege("UniversalXPConnect");
       ds.QueryInterface(Components.interfaces.nsIRDFRemoteDataSource).Refresh(true);
       ds.QueryInterface(Components.interfaces.nsIRDFRemoteDataSource).Flush();
       tree.builder.rebuild();
       }catch(e)
       {
           aggreg8dump(e);
       }
   }

5. Parsing the RSS.
   This was one of the most difficult parts of the project. Up until this stage in the development process, Aggreg8 was still using the server-side PHP RSS parser. Originally the plan was to use XMLHTTPRequest to download the RSS Feed but due to security reasons, this facility is only permitted for XML documents that reside on the same server as the script, which mean that RSS feeds on remote servers could not be downloaded. I spent a few days researching other ways of doing this and came up with the idea of a hidden Iframe. What Aggreg8 does is it downloads the RSS feed using normal HTTP download to an iframe in the display screen. This iframe has a maxheight of one pixel so it is virtually invisible. The Parser code, as shown below, then accesses the DOM of the document loaded in the IFrame and begins to extract the nescessary data using the javascript DOM function "getElementsByTagName". You can also see below that there is a check made to see what content type the document loaded in the Iframe is. If this document is not of type XMLDocument (ie. it does not have content type "text/xml") then the document will not have the desired DOM and will just be displayed as normal text. Aggreg8 is not able to parse such feeds as they do not have a valid content type. I have found that this is usually due to Windows Web Servers (IIS etc.) that do not have the content type of the documents sent in the header or rather have the wrong content-type sent in the header.

   var rssdocument = document.getElementById( "rss-frame" ).contentDocument;
               aggreg8dump(rssdocument.contentType);
               
               try{
                   if( rssdocument.contentType == "text/plain")
                   {
                       aggreg8dump("it is plain text , it should be xml");
   
                       contenterroroccurred = "true";
                       
                       aggreg8dump(rssdocument);
                   }
               }catch(e)
               {
                //var erroroccurred = "true";
                aggreg8dump(e);
               }
                   try{
                       var channelements = rssdocument.getElementsByTagName("channel");
                       var channelement = channelements[0];
                       var rsstitleElm = channelement.getElementsByTagName("title").item(0);        
                       var rsstitleText = rsstitleElm.firstChild.nodeValue;
                       aggreg8dump(channelements.length);
                       // .........................................
                       // this is just an excerpt from the RSS parser code

   There was another difficulty that arose when tryng to parse the loaded RSS feed. Aggreg8 would start parsing the document just after the the iframe was told what document to fetch. This caused problems as most times the document would not be fully loaded before the Parser would try to parse the data. To over come this a Listener was bound to the download process, and once the RSS feed was completely loaded , then and only then could parsing begin. below you can see the Listener code. This code was bound to the download and once fully downloaded would call the Parse function.

   
       // Get the RDF service
       var RDF = Components .classes["@mozilla.org/rdf/rdf-service;1"] .getService(Components.interfaces.nsIRDFService);
       // Get the datasource.
       var ds = RDF.GetDataSource(rssfeed);
       // Now see if it's loaded or not...
       var remote = ds.QueryInterface(Components.interfaces.nsIRDFRemoteDataSource);
   
       if (remote.loaded)
           {
           aggreg8dump("the datasource was already loaded!");
           aggreg8dump("now observing");
           
           try{
            Parser(rssfeed);
               
           }catch(e)
               {
               aggreg8dump(e);
               }
           }
           else
           {
           aggreg8dump("the datasource wasn't loaded, but it's loading now!");
           
           try{
           var sink = ds.QueryInterface(Components.interfaces.nsIRDFXMLSink);
           // Attach the observer to the datasource-as-sink
           sink.addXMLSinkObserver(Observer);
           aggreg8dump("now observing");
           }
   
   //Below ou can see the Observer code that listens to the download process
           
   var Observer =
       {         
       onBeginLoad: function(aSink)
           {
               
               aggreg8dump("onBeginLoad");
               
           },
       onInterrupt: function(aSink)
           {
               aggreg8dump("onInterrupt");
               
           },
       onResume: function(aSink)
           {
               aggreg8dump("onResume");
               
           },
       onEndLoad: function(aSink)
           {
               aggreg8dump("onEndLoad");
               aggreg8dump("done!");
               var progress_meter = document.getElementById( "progress-meter" );    
               progress_meter.setAttribute( "mode", "determined");
               progress_meter.setAttribute( "value", "0%");
               Parser(rssfeed);
           },
       onChange : function(ds, s, p, oldt, newt) {
   aggreg8dump("changed");
       
           },
       onError: function(aSink, aStatus, aErrorMsg)
           {
               aggreg8dump("onError");
               aggreg8dump("error! " + aErrorMsg);
           
           },
           
       };
       
   
   

6. Seperate files and package application.
   As described in the earlier (see 4.₂ on Chrome), the structure of the files in a Mozilla application is very important. All of the files in Aggreg8 are arranged in the manner described above. Meaning that packaging the application simply meant placing all directories, content, locale and skin in a JAR archive (using the Java JAR tool). This archive named "aggreg8.jar" and the install script , "install.js" (below) had to be added to a new archive named aggreg8.xpi (same as a JAR archive only renamed to XPI). Once that was all done Aggreg8 was a packaged application.
7. Write install script.
   Once an application is packaged into it's archive file, it needs an install script to tell Mozilla where to save the archive on the local machine. The install script must also update the "installed-chrome.txt" file in the chrome directory. Below you can see the Aggreg8 install script.
   
   
   
   

   const X_MSG = "Install Aggreg8";
   const X_NAME = "aggreg8";
   const X_VER = "0.0.1";
   const X_JAR_FILE = "aggreg8.jar";
   const X_CONTENT = "content/";
   const X_SKIN = "skin/";
   const X_LOCALE = "locale/en-US/";
   const X_CHROME = "chrome";
   
   var err = initInstall(X_MSG, X_NAME, X_VER);
   logComment("initInstall: " + err);
   
   logComment( "Installation started ..." );
   // add software to the installation
   addFile("We're on our way ...", X_JAR_FILE, getFolder(X_CHROME), "");
   
   registerChrome(CONTENT|DELAYED_CHROME, getFolder(X_CHROME, X_JAR_FILE), X_CONTENT);
   registerChrome(SKIN|DELAYED_CHROME, getFolder(X_CHROME, X_JAR_FILE), X_SKIN);
   registerChrome(LOCALE|DELAYED_CHROME, getFolder(X_CHROME, X_JAR_FILE), X_LOCALE);
   
   err = getLastError();
   if ( err == SUCCESS )
   { // if there have been no errors:
   performInstall(); // install "xfly.jar"
   alert("Please restart Mozilla");
   }
   else
   { // otherwise
   alert("there was an error " + err);
   cancelInstall(); // cancel the installation.
   }

   Once the install script is written and added to the application XPI, it must be triggered somehow. An XPI can be downloaded to the local machine and opened with Mozilla from there, or it can be run from a remote webpage. this link on the webpage will actually call a javscript function which will start the installation process.

   <!-- below you an see the code for the link on the Aggreg8 download page -->
   <a href="#" onclick="triggerURL( {'aggreg8' : 'http://www.philroche.net/redbrick/CA4Project/Download/aggreg8.xpi'});">
               Install v 0.0.1</a> {49kb}

   The code for the javascript function "triggerURL" called above by the link, is below. You can see that it first checks to see if that instance of Mozilla has permission to install application (there are permssion in Mozilla that can be set so that no new application can be installed). If this settng is not set, the installation process begins.

   <script language="javascript">
   <!--
   
   function triggerURL(url) {
   
   if (!InstallTrigger.updateEnabled())
   return false;
   
   else
   InstallTrigger.install(url);
   
   }
   
   //-->
   

   The installation guide for Aggreg8 guides the user through installing Aggreg8 automatically (using the afore-mentioned XPI ) or manually.

All of the source code for Aggreg8 is publically available on Mozdev's CVS server. Mozdev also have many ways of viewing the code form an application, the best is XPrint, which allows anybody browse the source code of a project without actually downloading, XPrint is set up so that you can view Aggreg8s code (http://xprint.mozdev.org/lxr/source/aggreg8/downloads/); The source code can also be downloaded in ZIP and JAR format from the Aggreg8 Download page. There is another document accompanying this one which gives excerpts of code important to Agreg8's functionality, see Code Excerpts Document.

As with all young projects, there is inevitably bugs found after it's first release. As Aggreg8 is hosted on Mozdev, Aggreg8 has the use of BugZilla which is a bug submitting and tracking tool. It is very useful and allows users of Aggreg8 to submit and follow the progress of a bug. { Aggreg's BugZilla : http://aggreg8.mozdev.org/bugs.html}.
Feature requests can also be submitted to BugZilla. There have been a few requests for features, including nested categories in the subscriptions tree. These features wil be added to subsequent releases of Aggreg8.

There were many hurdles (rather than problems) encountered while developing Aggreg8. I will list and briefly discuss the main problems and their solutions.

• Populating a Tree from an RDF datasource :This proved a huge hurdle. At the beginnning I had all the feeds hard coded in to the tree widget. I solved the problem by using XUL templates and rules (similar to that in XSLT).
• Sorting the tree into alphabetical order :There is a way of being able to sort the subscriptions tree alphabetically by simply clicking on the column headers. I had this working but when you try and select a feed in the tree, nothing happens. This is because, if the tree is sorted, the tree is moved to memory and the tree loses it's DOM. It is not possible to find out what feed has been selected, it is therefore that sorting the tree feature was dropped.
• Populating Edit Feed dialogue :This involved extracting the data from the selected feeds and placing it in the empty textboxes in the Edit feed dialogue on the subscriptions screen. I did have the subscriptions screen in an iframe in the XUL file but the file in the Iframe was unable to access the DOM of it's parent file to be able to get the data in the selected feed. I solved this problem by taking the file out of the iframe and placing it directly into the subscriptions xul file.
• Downloading RSS feed :As mentioned before, XMLHTTPRequest would not work , so I opted for the clever solution of a hidden Iframe whose DOM could be accessed by the Parser.
• Listening to RSS Feed Download :An RSS feed must be completely downloaded before parsing can begin. If the file was to be downloaded normally and Parsing to begin just as downloding began then an error would occur. I had to bind a Listener/Observer to the download process, and only when the download was complete could parsing begin.
• Parsing RSS 2.0 :RSS has many forms, there are three specifiactions that are used widely, 0.91, 1.0 and 2.0. It was very difficult to be able to extract the same data from each of these different specifications without Aggreg8 returning errors. I solved this problem by using a series of Try- Catch statements, catching the exception if an error occurred, and if an error does occurr another approach to the parsing is taken.
• Wrong Content-Type of some RSS feeds :RSS Feeds with a non-XML content-type can not be parsed by Aggreg8. This is due to the fact that Mozilla will not recognise the file as XML. It will recognise it as plain text, therefore it is not possible to parse RSS Feeds with a content type "text/plain".
• Installation on (Lin|Un)ix Machines if logged in as User :It is not possible at the moment to install Aggreg8 when logged into Linux/Unix as User unless you have permission to write to Mozilla's chrome directory. You will receive an Error - 215, which means that a required file does not exist. Installing Aggreg8 works perfectly fine when logged in as root. A workaround to this problem is to install Firebird (http://www.texturizer.net/firebird) and install Aggreg8 on that.

The learning curve I faced when I began this project was steep, very steep. I not only learned all about the Mozilla technologies and Framework, I also learned alot more.

• RSS:I learned all about RSS and the different specifications and forms it can come in.
• CVS:I have learned how CVS (concurrent versioning system) works and also how to use it.
• Namespaces:I have created a new namespace for Aggrge8 which specifies the Category name and feed Title etc.
• Open Source:I have learned about how the Open source community works and how projects are developed within this community.
• Lost Work:I have learned that work done that is not used in the project is not lost work, it is knowledge gained, the work done will always stand to you even if it is not used.

• Mozdev.org: Mozilla based projects are hosted on this site.
  http://www.mozdev.org
• Mozilla.org: Homepage for the Mozilla browser.
  http://www.mozilla.org
• Backend.Userland.com: A wealth of information for everything RSS related.
  http://backend.userland.com
• XULPlanet.com: Tutorials and examples of XUL.
  http://www.xulplanet.com
• Online Version of Creating Applications with Mozilla (O'Reilly).
  http://books.mozdev.org/index.html
• Netscape Developer Site
  http://devedge.netscape.com/

• Creating Applications with Mozilla (O'Reilly).
  http://www.oreilly.com/catalog/mozilla/
• JavaScript Bible (Goodman).
  http://www.dannyg.com/pubs/jsb/index.html
• Beginning PHP (Wrox).
  http://www.wrox.com

• Aggreg8.net - Homep of the Aggreg8 project.
  http://www.philroche.net/redbrick/CA4Project
• Aggreg8.mozdev.org - Homepage for the Aggreg8 project on Mozdev.
  http://aggreg8.mozdev.org/
• Aggreg8 Source Code on Mozdev CVS.
  http://www.mozdev.org/source/browse/aggreg8/downloads/
• Aggreg8 project proposal.
  http://www.philroche.net/redbrick/CA4Project/Documentation/proposal.php
• Aggreg8 Functional Specification.
  http://www.philroche.net/redbrick/CA4Project/Documentation/funcspec.php
• Aggreg8 Screen Shots
  http://www.philroche.net/redbrick/CA4Project/Documentation/Screenshots/screenshots.php
• Aggreg8 Installation Guide
  http://www.philroche.net/redbrick/CA4Project/Documentation/installationguide.php
• Aggreg8 User Manual
  http://www.philroche.net/redbrick/CA4Project/Documentation/usermanual.php
• Updates on Aggreg8 project.
  http://www.philroche.net/redbrick/CA4Project/Updates/weblog.php
• Useful links I have found that relate to the development of Aggreg8.
  http://www.philroche.net/redbrick/CA4Project/Updates/alllinks.php
• Useful RSS feeds I have found during the development of Aggreg8.
  http://www.philroche.net/redbrick/CA4Project/Updates/allrssfeeds.php

1. Mozilla.org where some of the material for the above manual came from.
2. Mozdev.org where some more of the material for the above manual came from.
3. XULPLanet.com where I got alot of the information on XUL.

Disclaimer: A technical document submitted to Dublin City University School of Computer Applications for the course CA400: Final year Project, 2002/2003. I hereby certify that the work presented and the material contained herein is my own except :

• Where explicitly stated references to other material is made.
• Where credit is given in the Credit section for; material written by or, for work done, by other parties.