I recently started a new job, in a new city. So those lesson plans I spent all summer putting together saw only one semester of use. Well, I hope my replacement finds them helpful. I've moved on to a federal internship. Not often one goes up from instructor to intern. But, in beginning this job, I've noticed a bizarre trend. I spent almost two weeks filling out forms, online forms. There are several different human resources programs; different systems for pay, retirement, health insurance, life insurance, and other benefits; at least four different systems for training; and none of these software packages or databases communicates with the others, or if it does it does so poorly.
It's been just a few weeks and already I cannot remember how many different systems I've been registered or filed in. I was struck, reading the user manual for one of at least four different document tracking and storage systems in place, how utterly nonsensical all of this is. Why have a system that is organized like this at all? And since we're here to discuss science fiction from a practical, engineering standpoint, let's ask the obvious question: how would a better system work? How would a system designed to utilize a computer system correctly and efficiently work? And ultimately the question I've been asking myself every time I learn anything about the modern US military: how would Starfleet do it?
Usually the answer is the exact opposite of what's going on here.
To understand how things are done now we need to go back in time a little. Back to a time many people can easily remember, and many of us have a hard time even imagining: a time before the personal computer. Go back just a few short decades and one finds a time when information was fasted transmitted by radio and further back than that by telegraph. In all times, from ancient Egypt to now, information could reliably be tranferred long distance by paper. Imagine you are in charge of a large body of personnel, say a Roman legion. You can't practically know everyone, much less have an accurate knowledge of events. So you tell your centurions to send you regular reports on their status. Of course you've got other concerns and can't always talk to them in detail, so you tell your centurions to write it all down. But some include too much information, things you just don't care about, others can't be bothered to even write down their own name or which century they are in command of. So you develop a basic format for them to use, tell them all their status reports have to have this set of information and be written this way. You give the reports to a clerk who writes down your legion's status in a log that you can refer back to in the future, and use to tell the Emperor or the Senate how things are going in your province.
This process and logic doesn't change whether we talk about the Roman legions, the East India Trading Company, the US Air Force, or the World Wide Wicket Company. You develop standard forms that ask for the information you want from those who should have it. Not only do you now know what you need, they also know what your need, and you have a written record for future reference.
Then we introduce the personal computer and every worker has easy access to a terminal. So instead of running off forms on a press and filling it out with a typwriter you can just do both on a word processor. Then print it out and file it. Later and better we improve data transfer and storage so you email where it needs to go and save a copy on a disk. Or better yet you store the relevant document on an common server where it can be accessed by whoever needs it. Later and better a dedicated software tracks the information in question, providing a virtual form you fill in and saving the information on a database that is then accessed by those who need that information.
Never do we ask why we're still bothering with a document at all. Throughout this whole process we've just taken the same system fundamentally used by the Roman legate and advanced the technology used to implement it, we never tried to see if the technology could replace it. At the beginning of this, after all, what we wanted was to get information from one place to another. Filling out forms was a means to that end, not the end goal. Since those forms are all we had to work with that was what we stored and used to access information. Now, however, we have something better, we have the information itself. All of the information, which we have decided to call "files", are saved somewhere on a harddrive as text. (Ultimately of course, it is stored in the form of physical charges, but that's a step further than we need right now.)
We don't need to continue treating the information we store on a server or harddrive as a paper document to be kept in a virtual folder. Doing so is a waste of disk space (which is, admittedly, cheap) and time (which is valuable). All information can just as easily be saved in a single place, to be accessed as needed by programs as needed. Instead we store all our information in discrete groups which are then accessed by programs once obtained. We even have systems for accessing those "files" that require checking them in and out, as though a book at a library! For what reason? We're talking about copies of information, everyone in the world could look at one of these documents without affecting the original. And to the computer itself even the distinction of what is and is not a "document" is arbitrary.
Why have almost a dozen different systems, each independent, each tracking one function? Why not have all relevant information stored on one secure server, with each piece of information accessible by the relevant system or personnel. Indeed a well written software system could unify all information access to provide only what is relevant to only those who should access it. Let's consider a personel file, as run by two groups. The first, as an example of how things are done now, the US Department of Defense, or DoD. The second we'll call Starfleet, always the best example of a better way to do things.
I mentioned before a bit how things work at the DoD, but let's walk through a little more in detail. A new employee comes in, the hiring process needlessly complicated in its own ways, and begins inprocessing. So first a personel file is created for him.
Then his pay information is collected, this is managed by a separate set of software in a separate department and is therefore stored in a separate file though it contains only two pieces of information not present in the personel file.
Then he tries to select his benefits. However doing so requires an account with a different system that manages benefits for all military personel and employees. Once registered in that system, which has no connection to the prior two, he is able to select benefits. His selections are input into the appropriate other systems, the (multiple) insurance providers sent his relevant information and the payroll system given his retirement information.
The pay system does not communicate directly with the schedule system, rather an intermediary system must be filled out by the employee each pay period to track time and hours. A different system allows him to track, monitor, and alter his pay information by acting as a go-between to the payroll system.
Now he enters training. One system tracks what training is required, another system allows the employee to apply for and schedule training, a third system gives access to that instruction which is available online. A fourth system provides standard, workplace safety instruction independent of the other three; which do communicate with each other, poorly. None of these three communicate with his supervisors or training department.
Another system, operated by another department, handles reimbursement for work related travel.
Two more systems give the employee access to shared documents within his department. Another two, independent of all the others, give access to work related documents regarding the project he oversees. Which is independent of the system used by the contractor to monitor and document the project.
Most (though not all) of these systems are accessed through a single common point that establishes the employee's identity, and he is registered in each one based on his degree of access to it.
Now let's consider Starfleet. Starfleet uses a single system to store and access all information on its computers, the Library Computer Archive Retrieval System, LCARS. That part I found on Memory Alpha, from here on out I'm making things up. From what we've seen it seems Starfleet has no civilian employees and those functions are performed by officers, and the Federation economy is another subject entirely, but for now we'll consider a similar situation as before.
A new employee comes in. Again the onboarding process begins. In Starfleet, however, only a single personel system exists. Stored in this system is all of the new employee's relevant information: identification, job, position, pay, insurance, and benefits.
The employee is able to access his own information, but no one else's. His appropriate payroll office can see employee's payroll information, but not their training or other personal details. All of this information is stored and accessed in a single, uniform system. LCARS knows where the employee is and how much someone in that position makes, it has his account information and work schedule, so it is able to forward his pay accurately.
LCARS has all the details on the job type the employee has entered, and therefore what training is necessary, and stores his completed training. LCARS also has all the virtual training courses saved and can therefore access all of them directly and track the employee's progress.
LCARS doesn't need to be a single, monolithic piece of software, but rather it is a single, unified system. Since all information is ultimately saved as text somewhere, why not save all that text in a single easy to access place? Each program accesses those pieces of the whole that it needs access to, each doesn't need its own dedicated system. Starfleet establishes a single standard for information storage and thus each program merely needs to be able to access that standard and know where the information that it needs is stored. LCARS itself needs only be able to identify what information each user is allowed to access and his permissions.
What a computer system doesn't need is documents. Why create a virtual version of a paper form to input information into a fictitious file to store information that can only be accessed by the same program that created it? Why have a dozen different versions of a document that was created on a dozen different file systems all relating to a single item? This completely fails to take advantage of one of a computer's primary purposes, it stores and accesses information. There's no reason to think in terms of files and folders at all, because that's not what's really going on. It is merely a metaphor, an extension of a system that should now be obsolete. We massively complicate things so that we can act like we're still storing all our information on microfiche.