Observing over the course of four decades how the world learned to speak in gigabytes and terabytes, I have experienced first-hand the evolution of the business of technology, the changing attitudes towards digitization, and the proliferation of data and its uses.
My adventures in data started in the mid-seventies at the Hebrew University of Jerusalem, where I could be spotted carrying a large tome, the manual for the Statistical Package for the Social Sciences (SPSS), the pioneering computerization of statistical analysis via specialized software for manipulating data. I would probably be on my way to the university’s data center, holding in my other hand a large stack of punch cards which I coded with a keypunch machine. Handing the cards to the computer operator, I prayed to the gods in the machine that I would not have to re-do the work, that I made no data entry or programming mistake.
In 1980, when my wife and I — both graduate students in sociology — were expecting our firstborn, I started to look for full-time work. There weren’t many (or any) jobs available for someone like me with a bachelor’s degree in sociology and education so I applied to basically every open position in Jerusalem, including a job as a supermarket assistant manager with the local supermarket chain.
To my surprise, after a few months (and still desperately looking for a job) I was invited to an interview with the CEO of that supermarket chain. At the time, the chain consisted of a dozen small neighborhood stores but also operated the largest supermarket in Israel in terms of transaction volume, located at the very center of Jerusalem. The CEO decided to install in that supermarket a minicomputer supporting a barcode scanning system and I was probably the only university graduate with at least some experience with computers among the applicants for the assistant manager position.
No other barcode scanning system existed in Israel at the time and there were no barcodes on any product, but the CEO, Simcha Rotem, wanted to demonstrate to (the very reluctant and skeptical) Israeli consumer goods manufacturers and retailers the benefits of adopting universal product. codes (this was not the first time he went to battle against all odds — as a 19-year-old he was one of the few fighters that survived the Warsaw Ghetto uprising).
I was hired to babysit the new computer system (a large refrigerator-size minicomputer from NCR hooked to about ten terminals / cash registers) and integrate it with the (very much not computerized) daily activities of the supermarket. In the late 1980’s, supermarket barcode scanning systems were a novelty even in the country where they were first introduced in 1974 — only 10% of US supermarket stores had installed such a system.
I had to organize from scratch the processes and procedures around the new system, taking into account the fact that this was a demonstration system in a country with no UPC codes on products. For example, ensuring that each product on the supermarkets’ shelves had a code on it for the cashier to enter into the cash register rather than entering a price as they used to do before and that the shelves displayed the correct price and that the codes were updated each time the price changed (which happened very often in Israel at the time — average yearly inflation was 120% in 1981).
This pioneering spirit extended to the two excellent NCR technicians that were tasked with the system’s maintenance after years of servicing traditional cash registers and a week or so of barcode scanning system training in Dayton, OH. One night they came to perform regular maintenance but neglected to first backup the system. The next morning, quite a few customers enjoyed paying lower prices as the system reverted to the price book from a few weeks or days before, the last time the system was updated. That was my first lesson in the importance of frequent and orderly data backups.
Even more important was what I learned about computers and data and the humans using (or getting used to) them. A supermarket scanning system was (and is), similar to any computer installed in any type of environment: It is a data collection machine.
In an early 1980s supermarket, “data” was limited to daily manual tabulations of transactions (adding up the numbers recorded on a strip of paper in each cash register), to be later processed by the accounting software running on a central computer. The new scanning system, the new constant and dutiful observer, collected a wealth of new data, including the number of customers and the number of items purchased by each customer every single second when the store was open. In addition, it provides data on the time it takes to process each item and the time that lapses between the end of scanning to accepting payment.
Launching my career as a data guru, I devised a formula to calculate the typical cashier workload in the supermarket based on 5-month worth of data on the number of customers, the number of items they bought, and the speed of processing them. All by hand, with pen and paper — the IBM PC was introduced in the US around that time, in August 1981.
I was probably the first person in Israel (and possibly one of the first in the world) that could tell you confidently how many cashiers were required at each hour at each day of the week, even predict the departures from the norm due to specific holidays or other factors affecting store traffic.  This workload calculation also applied to the manpower required in various store departments.
Before the arrival of the supermarket scanning system, staffing decisions were based on experience and intuition. Now, I could show the benefit of what we would call today “data-driven decisions,” accurately matching store traffic and customer activity with employee availability. According to my analysis, we were overstaffing the cash registers anywhere between 13% and 48%.
As often happens with pioneering efforts, this exercise in data analysis did not have any practical implications, as the management of the supermarket chain was not interested in data-driven decisions. The manufacturers of such systems at the time mostly talked about the potential reduction in employee pilferage as their key benefit.
I did a few things on my own, with the power vested in me as an assistant manager, such as ranking cashiers based on their machine-documented performance (speed of processing customers accurately, balancing the account, etc.) and posted on the bulletin board the names of the top three cashiers of the week. After six weeks of doing so, I concluded that the cashiers were paying more attention than before to their mistakes and how to correct them.
Another finding from my data analysis, one that serves me to this day when I need to decide which line to take in the supermarket, was that the time it takes to accept a payment is 2 to 3 times the time it takes to scan items. Given that “scanning” meant punching a product code in our unique case, this may or may not be the exact ratio with a full-functioning supermarket scanning system. But still, better take the line with fewer customers, even if their carts are full to the brim.
One obvious potential benefit of a supermarket scanning system was in managing inventory and planning the amount of space dedicated to each item. That was yet another important aspect of supermarket operations that used to be based on experience and intuition, and, when intuition failed, by observing and recognizing the complete emptiness of storage bins and shelves. I saw the potential of using our new data collection helper to significantly improve inventory management and the shelf-space allocation and positioning of individual products. 
My proposals went nowhere. After all, I was hired to babysit, not to play with data, certainly not to tell veteran supermarket operators how to improve efficiency, productivity, and customer satisfaction.
Assisting in managing the largest supermarket in Israel — and often actually managing all of its operational aspects as three different general managers were in charge of the supermarket during my time there — taught me a lot about managing people and about introducing computer technology, automation, and digitization into a traditional work environment. Just one example (and lesson): No matter how much I tried, I couldn’t make the head cashier understand what the computer system was spitting out as the daily summary of transactions for each cash register which she needed to total for the entire branch . But the supermarket’s general manager, with his thirty years of knowledge of how supermarkets worked, “translated” for her each line in the computer printout into the traditional numbers and categories she knew so well. When you move from analog to digital, first and foremost you must get the humans involved comfortable with the data and confident in managing it.
This experience helped me in my subsequent adventures in data and the evolution of modern computing.
At NORC, starting in 1984, experiencing the transition from large, centralized computers to personal ones and from a command-line to a graphical user interface; and managing the first-ever large-scale computer-assisted telephone interviewing (CATI) survey, yet another instance in which the analog — a lengthy and very complex paper-based survey in this case — became digital.
At DEC, which I joined at its peak in 1988, where I learned about the limitations of business data collection (“market research”) and the coming “convergence” of computers, communications, and media. And at EMC, which I joined in 1993, just after the beginning of its ascent, where I learned about the restructuring of the computer industry into “horizontal layers;” how to create new market segments (enterprise storage, networked-attached storage, and storage-area networks); and how to do “thought leadership” such as commissioning studies of how much data is created annually in the world.
Observing the world learning to speak in gigabytes and terabytes — and doing my bit to improve the general awareness of the role of data in our lives.
 In 1986, when scanning systems were already a common feature in most US supermarkets, a trade publication uncovered a rarity which dubbed it “America’s most efficient supermarket.” The owner-operator was reported to “chart by hand the automatically generated hourly sales and average customer transaction, duly noting the weather and any special conditions that might impact sales” (Progressive Grocer, March 1986, p. 114).
 The shelf-space allocation and product positioning idea became the conventional wisdom a few years later, helped by the growing number of scanning systems installed in the US (13,000 in 1986): “The trend reflects a fundamental shift in the balance of power between food. retailers and manufacturers… Armed with up-to-the-minute sales information from new computerized checkout-scanning systems, retailers can quickly pinpoint what products are moving, and at what level of profit. Today, retailers are calling the shots ”(The Wall Street Journal, November 1, 1988). This was an early example of how data can alter the traditional power dynamics in an industry.