![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
reddragdiva) wrote2015-12-03 05:13 pm
it’s tentacles all the way down pt dclxvi
My most and least favourite system that I have ever had to administer involved an embedded copy of OpenOffice.org (originally a Solaris SPARC binary of 3.1, later the Ubuntu 10.04 package of 3.2), started from a cgi-bin Perl script as needed, to convert MS Word .doc files to PDFs.
This is because MS Word .doc is such an utter shower (specification PDF, 563 pages) that it takes something approximately the size and complexity of MS Office to deal with it. And OOo is certainly that.
(We thankfully talked them out of making us install embedded MS Office instead, by offering to charge them the cost of a Windows sysadmin to run it. Never say no!)
The .doc files were end users’ edited versions of deeply defective source .rtf files generated from XML and XSLT by Apache FrontOffice, that wouldn’t open properly in anything else but MS Word. I wish I still had some, they’d be great bug fodder to submit to LibreOffice.
Even though this system was released in 2009, it never did manage to cope with MS Word 2007 .docx files.
It also kept said .doc files in a Subversion repo. Which was not operated using svn binaries, but twiddled by the relevant Java libs.
The cgi-bin Perl script was because the “generate PDF” functionality required a process fork, which hit a bug in Solaris SPARC Java 5 and 6 that tried to make a copy of the entire Tomcat process in physical RAM, which of course ran out of memory and failed. OOo won’t run if the user’s home directory isn’t writable, so on Ubuntu this required “sudo chown www-data /var/www“. Thankfully we weren’t using that as the webroot.
When I asked the (very good) senior developer about these design decisions, the pained expression on his face as he detailed why all these terrible ideas were the least-worst options was quite exquisite.
The purpose of a specification is to nail down the vague hopes and dreams of the business unit that’s just seen a fat, juicy market opportunity. In this case, they wanted a magical flying unicorn pony that ejaculated rainbows. The spec went into quite some detail about the desired properties of the wing feathers.
What they ended up with, of course, was a retired seaside donkey that had been tarred and feathered, a cornetto stuck on its head and fed food colouring and laxatives.
One part of the original specification of the system was to take updated versions of the source .rtf files and merge them with the user’s free-form .doc files such that the end users’ annotations would be preserved and stay correct. That is, the spec implicitly required the implementation of strong artificial intelligence. We told them that bit would have to be implemented later.
I quite delighted in deleting all traces of that system personally when, after five years’ soaking up money for two institutional customers ever, it was finally decommissioned.
Hear no evil, see no evil, 'spec no evil
Specification? What's that?
Picture, if you will, a bright and beautiful day in 2005, when a a sloppy contractor is told to organise a heap of spreadsheets into a departmental reporting system...
...Well, he's already made a good start on that by providing the data for these spreadsheets from an unstructured morass of SQL and SSIS jobs on a dev/test server running Microsoft SQL server.
So what he does next, is he inherits the license on an old Business Objects installation, on a server that nobody wanted but couldn't decomission because someone ran a legacy report on BO.
Business Objects is the flower of IBM-style corporate interface design from the mid-1990's. Only, without corporate-grade administration interfaces...
So... BO generates daily, quarterly, and hourly reports, for about 20 reports, and saves them in folders...
As MS-Excel files.
A web gateway is built in undocumented vbscript, in which a simple menu system allowes the users to download their familiar Excel reports.
So far, not so dreadful.
BO can also email out copies of reports on a schedule, and soon it is doing that. On a management console web page which does not support corporate-grade tools like listing users, overview of scheduling and the server load, and (say) labelling reports by sensitivity and integrating access/mailouts with Active Directory. Which is to say: once you go over a dozen users, you can't manage the accounts: it's easy to say "take me off report Foo, I'm moving to department Bar', if you know it's report Foo and you've got ten to twenty minutes to do it... But "Is user FuFu on your system?" and "List all of BaaBar's reports" is unknowable and unimaginable.
This comes back to haunt us, four hundred reports and ten years of user turnover later.
But the missing management interface that *really* bites us is the inability to list reports with undocumented VBA scripts embedded in them, generating secondary data files which act as essential data feeds for the SSIS scripts that load the dev database every morning...
Some of these files are feeds for unrelated systems. This, too, is undocumented.
...Secondary files that cannot be listed out: you can only know where they come from, and where they go, by knowing which of the hundreds of BO reports emits them, and opening up the VBA in Business Intelligence Studio.
You can see where this is going.
So, ten years of expansion and accretion and congealment down the road, over four hundred business-critical reports and undocumented feed files for other systems are running on an unsupported BO version on an unreliable server at 100% cpu load.
Reports running unmaintainable spaghetti SQL on the BO server, inside BO's incomprehensible 'data dimension' design framework, on top of a spaghetti morass of bad SQL, SSIS objects, and hundreds of tables in a SQL server on an unsupported 'Dev' box, reading data from dozens of undocumented data services everywhere in the company, with a critical dependency on dozens or hundreds of mystery files emitted by the very same Business Objects reports.
The SSIS jobs and the BO mystery file emissions are, of course, sequence-dependent and running on two different servers... One of which has a proper scheduler with an overview of what runs when, and the other one is Business Objects management console on a server at 100% cpu and a queuing problem.
The contractor who constructed it, expanded it, altered it repeatedly and without the slightest thought for design or consistency, and allowed it to accrete and worsen over the course of a decade, departed a year ago.
He left us a medium-sized spreadsheet documenting his opus magnum.
.
.
Re: Hear no evil, see no evil, 'spec no evil
This system I still posit as special because it started this way.
no subject
/usr/bin/soffice --headless --convert-to-pdf *.doc, eh?no subject