NSF replaced by XML? Nah. NSF enhanced by XML? YES!

Following on the heels of the MS Office XML announcement, Rich posted a good item about XML support for Lotus Notes and Domino, and Stan and Stephan and Philip Storry added their .02. 

XML is a great transport medium, but a bloated and cumbersome storage medium.  I used to think that relational XML databases were the wave of the future, but after working with a few and writing about the rest in the XML Programming Bible, I have to side with the relational traditionalists when it comes to file formats. 

All of the big relational vendors store their data in their own formats, and then publish bits and pieces to XML when asked.  This is not because it's hard to store it as XML, it's just not efficient or practical.  If you want reliability and performance, it's best to store data in a specialized format.   If you want a universal transport format that can be manipulated into just about any other format, then you want to transport it as XML. 

The announcement that the next version of all MS office documents will be based on XML actually gives us an interesting peek into the MS mindset when it comes to software.  Office file formats are not containers; they are a way of transporting information.  The file system (the Windows file system, of course….) is the container.  And that won't be XML.  Smart.

By contrast, DXL focuses on the design elements, describing containers, while ignoring key parts of documents.  The documents should be the focus, because like MS office file formats, they are actually transport mechanisms for data.  There is far too much design information in a DXL document.  DXL should describe the data, and let the container (Lotus Notes and Domino) worry about the method of storage.

My solution? Keep Notes the way it is, and focus on 100% DXL import and export fidelity with LotusScript and Java.  To start with, strip away all of the DXL stuff that describes the structure of Notes documents and views and just focus on the data. 
Next, create an optional process that publishes a light, design-free XML format for all documents to an external file system, and updates the XML data when the Notes data changes.   
Lastly, add an option that published Web-based data as XML instead of HTML using existing Domino Web processes, and allow it to be manipulated by an XSL stylesheet before it's published.

That way data that is not reliant on security can be published, and data that is published can be transported easily.  Notes gets to do what it does best, and so does XML. 

Oh, and I know this will work.  I have the code for most of this. I've been working on it since 1999 when I co-authored XML: Powered by Domino.  I published updated versions in the Notes and Domino 6 Programming Bible a couple of years back, and have been enhancing it ever since……

Comments

[Andrew Pollack]

Brian, I don't think any xml/rdb story is the end point. I really think that a hybrid solution is ultimately what most applications will be built of.

Particularly, I see traditional RDB for raw data and for storing indexing meta-data about the target content item itself, with various JSR-170 compliant storage solutions (including, I expect, Domino) being used to store the unstructured or semi-structured INFORMATION (as apposed to data). This information may be in the form of XML as it is by its nature going to be self-focused. By self focused, I'm again contrasting it with mass-focused raw data. A single time & temperature reading is of little real long term value. A mass of time and temp readings from a site or many sites over many years has huge value. That's mass-focused. A report on the efficacy of a new drug is a single entity and contains value within itself (self focused) and thus makes a good target for storage in a jsr-170 document base, and may be a good thing to store in XML if cross application readability is important.

-- Ah well, that's my sub blog response.

[Richard Schwartz]

Brian,

I think we agree more than we disagree. I'm interested not just in publishing though. I'm interested in making it "feel" like native access, both for reading and writing. 100% DXL import/export would be a nice side-effect of what I'm suggesting. We'd get that as an on-demand capability even in databases where we choose not to adopt native XML storage. (In fact, I think the only way we'll ever get to 100% is if IBM were to take on something as truly ambitious as native document storage in XML files.)

-rich

[Stephan H. Wissel]

Eventually Brian and Richard are not soo far off with their optinions. Domino can store data internally data in whatever format it wants to as long as it is easily accessible via XML (see my ideas/pleas in the Notes6 and Notes7 Beta dbs for that). The idea to see notes documents as XML files in the file system is intreaguing (that's probably spelled wrongly ). AND it's just a little step when you take a wider "file system" definition: enter webDAV. Windows and more open OS can mount a webDAV "directory" as part of their file-system. webDAV is nothing more that some additional methods. It should be an easy exercise to write a servlet (that even would run on the Domino servlet container) that does exactly that.
This way:
a) runs on all Domino
b) runs on all os
c) provides a native XML interface as files
d) uses the Domino storage
... expect something on fromDomino soon.
:-) stw