Perhaps an application tool that could be used in conjunction with web site development and not as a external server service.

As the tags are extracted and deleted from the text, subsequent tags offsets change. This can be solved by working backwards when extracting tags, and forwards when reinserting. Not too trivial, but a nice algorithm can be found.

However, editing the plain text independently afterwards would be impossible, especially if the text were modified in several places! It is questionable to have to rely on fragile resynchronization – if there were a corruption in the database, then the presentation could turn to goo!

So, for work in progress, the tags and text would have to be reintegrated for editing… which means that you might as well have stored the raw html in the first place! If you need to export the pure text, then a simpler tag stripping routine will do on demand…

However, your idea has merit if you want to do sql searching, and don’t have enough space to store a parallel text only field.

I think the bottom line for speed and effectiveness would be just to store text only + formatted text in separate fields. This may double storage requirements and be less elegant, but response time should be quicker.

Regards, Andy.

No need to be sorry.

The thing about this is I am not talking about a full HTML document. I am talking about a small part of the page (for a lack of a better term). I am not convinced that transforming HTML -> XML/XHTML -> XML+XSLT -> HTML is really that great of a solution. That is a lot of CPU work to just display formatting. And again, I am not talking about structured data per se simply formatted data. I know that solution is the standard one, but it seems like a bit of overkill when dealing with very formatting on small “pods” of information.

If you are using a text area to capture data it is likely that this data is in some way structured (e.g. paragraphs, word emphasis, lists, etc.) and this structure is what HTML is designed to represent, so why strip the data (text) of its structural meaning?

If it is to ‘repurpose’ the data for non-html display, why not use XSLT to transform the data into the required format: for example, simply replacing P tags with /n (newline) or more complex XML transformations such as to pdf or MS new XML document format.

If it is to enable easier full-text searching then it may be possible to leaverage the database to index the structured data for you? Of course not all databases are equal, but we use SQL2005 and it allows full text indexes of XML data. If this is not possible, then perhaps Peter’s suggestion of an additional non-html data column in the database might suffice.

It seems to me that you lose more my striping the data of its structural meaning than you gain. Sorry.

Regards

But, I mean, for stuff where there isn’t a meaning per se. Think of this as the text in one tag of an XML document – or in a CDATA section. If someone adds bold or italic or underline to the display of content, there really isn’t meaning to that just emphasis or style. While I’d like to think people will, say, italicize or tag something as “book name”, odds are they wont.

“have you thought about structures like paragraphs and unordered lists?”

Not yet, but I think those kinds of things are more data centric – this thought is not completely flushed out yet mind you – I think that might need to be handled differently… not sure yet. But it could work the same way as the bold italic example. I’d like to try to find something before I go and build it :)

“I don’t see anything wrong with storing data in the database. I see the html markup as data. It’s no different than storing an image or other binary data in the database. “

That’s true, but I don’t often search though image data to find matching results nor has anyone ever asked me to give stats on the stuff inside of an image data field. I do see your point, but I don’t think the comparison between binary data and html formatted data completely correlates.

“I’m not sure how easy to implement something like that would be, plus I wonder how worth it it all is?”

If done correctly, in the end, it should boil down to 2 functions – “text[] styleSheetFromText(text)” and “text applyStyleSheet(text,sheet)”. The details of those might get a bit hairy. As to how worth it it is, that depends on how clean you want the data in the end, and if you think it’s a good idea – but I wonder too :)

“I did once read about a text format that stored formating info in the white space at the end of lines – intended for newsgroup use, iirc.”

Hum… sounds pretty close. Do you know what it was called?

“Page.HTML with markup and Page.Text without, stripping all tags from one (an easier proposition)”

Thanks Peter, that’s not a bad idea. It essentially doubles the storage, but will mostly get the job done. If I can’t figure out a nice way to separate the two, I’ll fall back on that.

(My apologies for the delays in showing your comments, I have to moderate the comments because ever since I moved to WordPress the spam comments have quadrupled.)

Since the markup was valid XHTML then parse the XHTML into a rendered plain text view. This turned out to be more cumbersome than I hoped–just because there aren’t any good open source Java solutions for converted XHTML to formatted plain text, so I had to write the parser myself.

However, this allows us to store both plain text and XHTML versions of the user created content. Obviously, the plain text code doesn’t include images, but it allows us to send documents to mobile devices w/out having to worry about whether they support HTML.

On the other hand, given that disk space is cheap and programmers are not, one solution would just be t have Page.HTML with markup and Page.Text without, stripping all tags from one (an easier proposition) and using the text version for some things and the HTML for others. Wouldn’t have the full utility of having access to markup semantics to translate between formats (but of course that wouldn’t be of use until you’d written tools to go from the language to PDF or whatever).

Definitely an interesting concept. Let us know how you get on!

It is an interesting area though in regards to separating presentation from structure and how that works when using a database. It’s one thing to work with static HTML pages using an XHTML file and separate CSS file, but when you start drawing in dynamic content from a database it adds a different angle to that separation.

Personally I don’t have a problem storing the XHTML in the database. I would have though that good semantically structured XHTML mark-up is the only thing (ideally!) that would be getting added to the plain text. Therefore it should represent no problem simply stripping that mark-up from the content drawn from the database leaving you with the raw text?

Perhaps the new Wysiwyg editor WYMeditor ( http://www.wymeditor.org/en/ )could be useful for keeping people to using good mark-up?

Peter Krantz wrote an update to his ‘Evaluation of WYSIWYG editors’ post which is interesting reading. http://www.standards-schmandards.com/2007/wysiwyg-editor-test-2/

Anyhow, it seems like the effort to do what you’re suggesting is disproportionate to the rewards of the effort. Why would you actually use this? About the only reason I can come up with is if you wanted to put the text into a format that’s not HTML. IE: flash or something else similar.