Middleware

Modern software operates within the Cloud or internet. Software requires its own software infrastructure. Some code runs on the Cloud or internet servers. Some code runs within the user’s browser. These languages are different. The back-end software tends to be structured procedural languages. The client-side software tends to be written using object oriented languages. And what runs between the client and the server?
The first time I heard the phrase middleware, I was writing software for FedEx with a terrific team of Oracle developers in the late 1990s. I had to ask for a definition. In short, middleware is software that run between two things. Middleware is software that is invisible to the user. Middleware is software but seems to fall outside the accepted definitions of an application or app. It is software’s own software-based infrastructure. Does that make any sense? Probably not.
I wrote my first lines of code on a PDP-11 microcomputer from Digital Electronics Corporation (DEC) during high school. I attended a school in a wealthy area surrounded by the massively burgeoning IT industry.  In my high school, four years before the IBM-PC came to the market, I learned BASIC. The name literally meant “Beginner’s All-purpose Symbolic Instruction Code”, and no does not relate to the American slang basic, which seems to be a bit of an insult. 

10 BEGIN 
20 PRINT “HELLO WORLD”
30 END

Hard to imagine how these building blocks gave us the world of modern digital commerce. Those roots go back even farther. That trip requires tipping our hat in honor of Ada Lovelace, Alen Turing.
My educational progression followed the global development of software and software tools. In 1982, heading to university, I bought an IBM-PC for $2,500. That was the discount a university professor got. Adjusted for 2022 dollars, the cost would approximate $7,500. 
As a college first year, I thought I would study computer science. Regrettably, my university’s systems were older than my high school’s systems. No way was I going to learn how to program in assembly language on a huge mainframe computer. Mainframes, in 1982, were already doomed, so I thought. I never wanted to write an operating system, which was one of the capstone assignments. 
I finally took CS-111 as an independent study. In a class by myself, I read Cooper and Clancy’s “Oh Pascal” learning the software language Pascal. That book remains on the shelf behind me in my office today. 
At the age of 28, I had already been a contributing author and technical editor for books about computer programming. Two evenings a week, I stood in front of a group of adults teaching programming skills at a community college. One high school elective and single independent study course for one semester started me on my career. While an autodidact, nearly all I learned fell beyond the reach of classrooms. School never came easily to me, likely because of my learning differences. 
I watch colleagues like Dimitri in awe. In direct comparison, I see that I fraud and the idiot, even today. We’re all like that though, aren’t we? We get good at something, or we get recognition then we tell ourselves: No, I don’t deserve this. From my perspective, he is better skilled that I, smarter than I. I admire he jumps between programming languages and environments.
I mentioned Pascal on purpose, not just as the rambling digression (I do love my digressions though). The programming language I use for work today had been built from Pascal. Oracle, when needing to create a procedural programming language borrowed heavily from Pascal. Pascal is an imperative and procedural programming language, a natural progression for me from BASIC. The language had been designed by Niklaus Wirth. Mr. Wirth, who is 88 years old in 2022, won the Turing Prize in 1984, roughly the same year I learned Pascal. 
Stevie and I write code in Oracle’s PL/SQL language. The language derived from an earlier structured procedural language called Pascal, that I learned at university after learning BASIC in high school. We declare variables at the top of a subroutine. We write code in logical subroutines called procedures or functions. These get compiled by Oracle into the database as something that is no longer intelligible to human readers. These routines / subroutines form building blocks within the database to perform tasks, typically with data. One might need a routine that calculates the total value of an invoice. That routine must spin through each line of data for an invoice. The routine identifies the quantity of items ordered. It then multiplies quantity by the unit price to generate the value of that invoice line. Then we tally the total of the invoice lines to get the invoice total. We, the developers, must control this process precisely, due to the variations needed. When do you round the numbers to two decimal places? When do you calculate the taxes? How do you handle items that are not taxed. How to you calculate discounts? Each of these steps must follow the client’s instructions and we must do it precisely. The process and math must be consistent and transparent. Someone will run a calculator through the numbers to confirm the math. Later, an auditor will verify every bit of it. Nothing can be hidden. 
That’s the land of managing the boring business data: customer contacts, invoice information, inspection data, reporting, document management, bank balances, reservations, etc. My world.
Our software depends on a user running the application within a browser. The user updates a customer’s profile or generates a service order. Those tasks such as presenting the data entry web pages to the user’s browser is accomplished by Oracle APEX. I do not have to write the JavaScript and HTML and CSS required to make an application operate within a browser. 98% web-based applications rely on JavaScript. 100% of web-based applications depend on HTML and CSS. We tell the browser to paint a region blue by setting the region’s color property to blue. We do not tell the browser how to do the painting. We only specify the color. We don’t tell a browser how to draw a line, we specify the thickness and style of the line. I won’t present a sample of object-oriented code to read. It doesn’t read well, maybe it is hard to hear: onclick= “void(0);”
When I look at our applications written in Oracle APEX, the apps can look different on the different browsers. Occasionally, in Firefox, I see fonts with serif instead of the sans-serif font specified by our team. These variations are reminders of a key difference between procedural languages and object-oriented languages. With object-oriented languages, such as JavaScript, the browser decides how to execute a process. The browser’s own personality expresses itself a bit. 
One might argue that the process of formatting and presenting data to users via a browser could be called “edge-ware”. Browsers and JavaScript is software sitting at the edge of an application. Edge-ware, a term absolutely nobody uses, sits at the edge of a software application formatting and presenting our pages to the user. Ok, if you do an internet search on Edge-ware, you will find a village in New Zealand and a couple of companies. Let’s also quietly acknowledge that JavaScript is used on servers too. That is a more recent development. 
The user logs into our Oracle APEX application via browser. The data travels back and forth between the user and the Oracle server via the internet. For illustrative purposes, the Oracle Server sits in the middle of the internet, the “Cloud”. Users sit at the edge of the cloud with their phones, tablets, laptops, and desktop computers. And let’s add cars, refrigerators, doorbell cameras, modern televisions – these all sit at the edge of the cloud with you the user.  
The spirit of “edge-ware” tells a story. 
If we have lovely object-oriented software running on the client-side, or edge, of our applications. And we have lovely software written in a procedural language compiled within the Oracle database, then how does the client/edge talk to the server and the database? Say “Hello” to middleware.
Middleware is like a Babelfish (for fans of Hitchhiker’s Guide books) or the Universal Translator (for fans of Star Trek). Middleware does this cool job of shuttling instructions and data between two very different environments that use different languages and different technologies. 
I think I defined middleware. It required 2000 words to accomplish that goal. Through the journey I did a bit to differentiate structured programming languages from object-oriented languages.
In December and January of 2021, our project with Electrotest had been defined as middleware. With a careful stretch of my definition above, our software could, maybe, possibly be called middleware. The project leaders used the term as a means of evaluating the replacement of multiple legacy systems. Our software would be a central repository for data from several external and legacy systems. Our software would sit in the middle of existing systems. Fair enough, close to the definition above, without all the discussion about babelfish or universal translators. I suspect that the project leaders used the term as a way of exploring writing new software without committing to the project. Likely also as way of softening the discussions with folks who may be resistant to change. 
As a team, we had been given the opportunity to solve some long term and systemic issues. A CEO rolling through announcing massive project to all software that helps the company do its thing would be unpopular. These projects occasionally fail or people get resistant to the change. 
The project leadership used the term “middleware” to help others accept these early steps. Middleware sounds less threatening than: “Our software is crap, we’re replacing our crappy software for the unknown crap written by someone else who doesn’t now crap about our wonderful business.” That’s good political thinking and good leadership. 
We’re going to stick this reliable, awesome Oracle database into the middle of our systems to help solve problems our other software doesn’t solve. 
To the user community at Electrotest, this approach appeared friendly and hopeful. Everyone recognizes problems in systems. We’ll just fix those in a targeted manner.
To the new developers, we get the message, if this goes well maybe we can take more on later. Let’s get one thing working and evaluate.
I need to remember this trick. I have been in several situations during the last decade during which the big boss stands up in front of the users to mandate immediate and instantaneous adoption of our software. The big boss writes a memo reinforcing the message. Within months, the resistance from management overwhelms all work. Resistance to change kills custom software development. And please don’t ask us programmers to manage that for you. 
Electrotest’s approach likely generated an environment of collaboration and success for both the user community and the developers. This definition of “middleware” became more political than technical. It worked. Well done to the team. I’d never had conceived of that approach. 
We benefitted from viewing ourselves as this fake-middleware. We reach into the other systems to review their data whilst exploring past mistakes. We did not need a lot of input from user for this analysis. Hundreds of customers had been named with a dot, or two dots, or a slash, or two slashes, or three slashes. What can I infer from that? 
First inference is that the legacy system, DAX, required the user to type in a name for the client. But the client’s name data entry process had nothing behind the scenes controlling it. Second, I inferred that most users were right handed. Their favorite keys results from movements of their right hand. Both the dot and slash sit below the right-hand pinkie finger. Why were they doing this? Clearly, I can’t know without interviewing the users. I make more inference:
  • The user perceives no value in entering the data
  • The user does not possess the data and therefore can’t do the data entry
  •  There is something about the system causes the user to curtail their efforts
One observation I made is that their unique identifiers for service orders were limited to six numeric characters. Meaning that each time they got to their millionth order, the order numbering process started over from 1. When you do that, the order number is no longer unique. It appeared to me that they cycled through their order numbers every two to four years. When unique identifiers fail at being unique, the data is junk. 
Another observation about the data is that some orders had a value of one cent. Some had a value of negative one cent: zero point zero one euros. Again, I inferred the users need to perform operational tasks that the system did not understand. The system required a price on each line, therefore zero could not be permitted. The users had to create order lines that had zero prices.
The users were not wrong, the software was wrong. This happens. We call this opportunity. Business owners likely call it something else.
What did we have to do to prove we could make a few improvements? I believe we had to demonstrate our ability to write software that met their specifications, solve problems, and do it at a reasonable cost while executing quickly. When managing corporate data and building an ERP (Enterprise Resource Planning) and or CRM (Customer/Contact Resource Management), customer data forms the center of the data picture. One cannot create a service order nor an invoice without knowing who the customer is and their preferences. 
We had hundreds of pages and menus in two applications that were linked. We included over one hundred thousand rows of existing corporate data. We had a reliable and proven login process for user authentication and authorization. When the client looked at the system roughly six weeks after telling us to execute, they saw something that they recognized as their system: their colors, their data, their flow. We wrapped it neatly in corporate branding with their logos and security. It worked reliably on Windows computers, Apple computers, tablets, phones, and desktops. It worked on Safari, Chrome, Edge, and Firefox. 
Some of the menu items took you to page 1, the empty dashboard page. Actually, most of the menu items took you to page 1, the empty dashboard page. What do you want in six weeks? The customer profile stuff looked good enough to earn credibility with a new client. We did put the server’s time on the dashboard. The server runs in UTC, the coordinated universal time sometimes called Greenwich Mean Time or Zulu (that a NATO and military term).
We wrote almost no database code with PL/SQL, our procedural language. We limited our effort with client-side languages. We designated the desired fonts and corporate colors using HTML and CSS as appropriate. We did not have to do this for hundreds of new pages. We did it once and we were done (for now – we kept coming back to the look-n-feel, corporate colors, and logos). We never touched the middleware between the client’s browser and the server’s Oracle environment. 
For the dozens of lookup tables, we made one, the copied it over and over and over. All of the lookup tables had roughly the same structure with the same fieldnames. With the exception of the primary key, which was unique for each table, all of the look up tables were the same. 
We did not hit perfect. We could not achieve perfection. We did not have formal user requirements. We used our experiences and expertise to build a system we thought would meet their needs. We did not then have direct access to the client to ask questions. We did our work based on our professional standards. That’s what we presented. I am certain someone said: “Oh that’s perfect” or “Hey terrific”, but within days we got requests to make changes. Not perfect, but good enough. Sometimes good enough is good enough. When the mission is to build something fast, grab a big hammer and start swinging. There is time later to do the finish work, efforts we call “painting with a small brush”. 

Keep in touch

Notification of new episodes and maybe something else

checkmark Got it. You're on the list!
2020 Fijre Media LLC