Subscribe to DSC Newsletter

I recently noticed a proliferation of websites from organizations in my area openly offering to investigate ghosts. I thought it would be interesting to share my peculiar data-oriented slant on the subject. It is possible to approach the issue of ghosts from different perspectives. I have been taught to accept the existence of at least one ghost as a result of religious upbringing. I also support the idea of "transient" phantoms. Like an algorithm expanding a compressed file, the multifarious tendrils of fate operate in the background. Regardless of how complex a transformation might be, there are faint and sometimes imperceptible events leading up to the conclusion. For me although I am not a Calvinist, ghosts roam about in these situations of pre-destiny, functioning as agents of conclusion. I will be returning to this line of thought a bit closer to the end of the blog. At the moment, I intend to discuss ordinary ghosts - those that haunt people and that seem to attract the attention of private investigators. Being haunted represents a perspective that might be described as the "lived experience." Ghost stories are almost inseparable from the lived experiences of those involved. Without detracting from or questioning the relationships that ghosts have with people, I will be steering the discussion here more towards the idea of "haunted structures." Ghosts as they exist in common fiction are routinely associated with fixed and sometimes portable structures especially after the structures have been influenced by supernatural forces. When storms brought disaster to the city of Tacloban in the Philippines, apparently some of the locals started to see the bodies of those that had perished wandering in the streets at night. A city is one kind of structure. Other types of structures include emergency services, community support, and the system of politics responsible for the mobilization of resources.

The "stress response" is an idea from stress research. It suggests that stress is something internal rather than a product of the surrounding environment. Things in the environment that might lead a person to feel stress are called stressors. Just for the sake of intellectual adventure, consider what a "ghost response" might be or how it could manifest itself. The idea of a stressor is an interesting concept all on its own since certain structures that exist in society are persistent. Meanwhile, people pass in and out of society - sometimes physically through life and death but also socially through engagement and alienation. If the transients feel stress irrespective of the individual, and the environment is persistent, it seems reasonable to suggest that the latter contributes to the former. Imagine that structures in society can indeed trigger some kind of embodied "ghost response." Consider the possibility that a subway system might be haunted by spirits. It is a terrible line of discourse given that ghosts are normally associated with dead people. I would like readers to consider stepping away from the idea of haunted people - or people haunting others - in order to think about structures being haunted. We tend to associate haunting with the supernatural. But I portray it here as something data-symptomatic. The means of conveyance - the data - starts to collapse and behave abnormally when the circumstances of its use go beyond the intended parameters. Hopefully this will make more sense in moment.

Fellow travelers in life sometimes feel detached from society. Meandering over the awkward roads of existence, we do not necessarily do anything about the harshness and difficulties that others face. Many people prefer not to dwell on misfortune. However, when death occurs, the dead are supposed to stop registering as data. They should not be generating anymore data. We might keep a registry of people passing on. However, most of our metrics, recording systems, and databases are designed for living people. This makes a lot of sense since a dead person would not normally be expected to change state, situation, or condition at any point in the future. Yet living people, similar to those that are dead, sometimes carry on their lives in silence. They fill the environments that often remain long after they die. One wonders whether environments exist for these people or whether the people exist to occupy the environments. It's an interesting discourse in that the metrics that we have about people might in reality be measurements of their persistent environments. The focus of existence might not be in the person. Perhaps we are merely accessories or transient passengers for the environments we occupy - like bacteria living on a host. If something physical and tangible can be haunted, then our environments and the structures within it might be ghost habitats.

There are of course many different types of structures. In most ghost stories, structures have physical properties. It might be a haunted closet, chest, or basement. I have seen television programs containing haunted vehicles that terrorize motorists and pedestrians. Given all of these persistent structures around us today, there are many more opportunities for things in our lives to be haunted or be associated with ghosts. I just want to emphasize that I haven't addressed what "haunting" actually means, not yet. I said that data as a means of conveyance might start to collapse. But what does this have to do with being haunted by ghosts? I am just exploring the notion of the haunted structure. I recall going down a particular subway station - when I used to take public transit - and sometimes finding the platform empty. It seemed kind of eerie being in a facility meant for lots of people - and finding just myself and a few others. This emptiness is apparent in the physical capital - the walls, stairs, and floors. This vacancy is also evident in the data, which as I have pointed out is normally generated by living people. In effect, the systems in place to serve us are meant to measure the extent of our presence.

What Ghosts Seem to Do

In most television shows and films that I have seen, ghosts seem to have a need to do the following: 1) say something; or 2) set something straight. I personally recognize haunting as a form of community activism. An injustice has to be addressed. A voice must end the silence. I prepared a "ghost story" just for this blog. In this story, consider the "structure" of the group. For many years, a group of school girls would go to a corner store owned by a gentle old shopkeeper of poor health. The girls often shoplifted. They vandalized the store. They also ridiculed the owner. They sometimes complained to patrons in the store about being harassed by the owner although this never occurred. One day, perhaps due to the stress of dealing with the girls, the owner suffered a heart attack. He passed away. The store was closed down. This is when the hauntings started. Locals started to report strange noises from the building where the store once stood. Some said that they saw the shopkeeper sweeping at night. The girls talked about feeling the deceased owner's "presence" among them. A purely psychological perspective might attribute these feelings to guilt. However, in the structure of the relationship between these girls, the presence of the owner is or was extremely important. He helped shape their interaction; he gave them something to talk about; provided common threads and reference points; influenced their daily activities. The emptiness created by the death of the owner could be felt by the girls. They could feel his absence most when they were together as a group. Part of their collective identity existed "for" the owner.

Many people are horrified just by the thought of ghosts. It seems impossible to escape a ghost without confronting the question of what brought about its existence. It is through data and our focus on all things tangible that the existence of ghosts is delegitimized. It is not necessarily that ghosts do not exist but rather that the data does not take their existence into account. Since data is designed to express only limited spectra of reality, I personally consider it a poor source of guidance in relation to ghosts. In order to substantiate the non-detectable, it is necessary to set aside the authority of detectability. Things can exist outside the contextual domain or scope of our metrics. In my ghost story, the girls intended to harass the shop owner; they created a toxic environment for him. I routinely discuss in my blogs the idea of invisibility or lack of inclusion in data: this can happen when the metrics are designed to detect a specific aspect of reality while omitting the broader truth. For example, through a process of "projection," an organization can fabricate the world it chooses to occupy. While the girls in my story might continue their lives as if nothing had happened - the materiality of their existence being unaffected - in fact their "structural" situations have radically changed as a result of the death. Any presumption of disassociation from the store owner is most certainly short-sighted. The girls remain intimately connected to this dead individual; they simply never recognized his importance and involvement in their lives. His tangibility is not diminished by his demise. It is not life or death per se that shaped their relationship with the shopkeeper but rather his victimization and more specifically the relentless desire of these girls to make him a victim.

I came across an article recently about retail property owners being haunted - by the departure of Target from Canada. Large retail spaces are difficult to fill. More to the point, the business model, the retail spaces, and income stream are all premised on customers being in the stores. The data systems and business metrics are designed to measure and monitor living people. The absence of these people - whether it is by death or maybe due to better deals elsewhere - can be "felt" by these structures created to interact with and handle customers. The structures can be haunted by dead people, in a manner of speaking. Absence of presence - that is to say, the movements of disembodied spirits or non-bodied things - is not supposed to result in business guidance. Customers are not supposed to register as data. However, this is only so in an instrumental scheme. "We will put products on the shelves. Customers will buy the products. We will collect revenues. The benefits will trickle down to shareholders." It almost seems like a turnkey operation. But if the store never listened to customers while these customers were alive (i.e. present), and the store continued to ignore them as they died (i.e. disappeared or departed), clearly the business was never designed to handle real customers. It is not so much that the patrons are "haunting" the empty retail spaces; but rather, the facilities are haunted by the absence of patrons and more specifically the failed attempt to bring about their conformance to the business plan.

Out of respect for those that have shared actual ghost stories with me, I don't intend to offer any real-life cases involving individuals. I would say however that there is sometimes an unwillingness to see things as they are; because it is necessary to reconcile the harsh reality against how things should have been. When large amounts of data are systematically collected, it is important to note how the framework is often designed to deal with "how things should have been." How they were expected to be. The system might not be sensitive to failure; indeed, there might be no contingencies to deal with an "unexpected" reality. Then we feel the absence of people - an absence the data was never meant to monitor. The systems or structures in place become haunted, replaying the spectre of opportunities lost. It is true that people sometimes feel the presence of ghosts. Why not? People feel all sorts of things. They can feel hungry, lonely, lost. But it is the structures that become haunted. For those girls in my fictitious ghost story, their group structure was haunted; the death of the store owner brought about the realization of the group's pathetic preoccupation.

Consequently, for me haunting is related to data alienation. Lack of recognition or sensitivity to the reality that exists creates an illusion or fabrication. When the structures built over these flimsy shards of truth collapse, there is the "ghost response." The structure becomes haunted. That's scary. I once read of mythical creatures sort of like a people that can separate the torso from rest of the body; it then becomes possible to see all of the exposed guts and internal organs around where the belly would normally be. Some might say, hey, that's a dumb scary creature that doesn't really exist. When ghosts emerge to have their say or make their peace, it is sometimes necessary to acknowledge how they lived in silence, not really participating in the data that was collected. Feelings of fear and apprehension might rise upon failure. This means that organizations on the road to failure might even now be building up worthless databases full of fluff and nonsense, providing fuel for hauntings yet to come. The market leaders that some companies believe themselves to be might likewise be creatures that don't really exist. Finding creative and truthful ways of articulating the needs of the market helps to keep the data aligned with the business environment. Being sensitive and responsive to the howls and cries within the organization helps keep the data grounded to its production realities and capabilities.

Ghosts in Structural-Organizational Pathology

I remember reading a journal article about how a species can be extinct while still being alive. A species might already be doomed to extinction although there are a few members remaining to live out the balance of their existence. The species will never recover if the population isn't large enough to overcome natural attrition. Dying as the last of one's kind certainly leads to a truly horrific death, life being so pointless on many levels. Since reading the article, I have been fascinated by the idea that the "appearance of success" might sometimes be a terrible illusion; that doom might be waiting just around the corner. One should wonder about the forensics and pathology in the data, how market termination can catch a company "by surprise." Even more engaging is the proposition that a company might lead itself down a happy-go-lucky path - only to encounter forces beyond its control sometimes characterized as ghosts. I say they are ghosts because they occupy a realm outside the substantive layer of discourse of an organization - it seems at times beyond the detection of managers and executives.  Being outside the realm of discourse, the word "ghost" might not be used; in fact, there wouldn't be any words used at all, being outside the substantive radar.

I find it disturbing and rather interesting how difficult it seems to be to detect ghosts - until it is too late. It takes systematic subterfuge, sensory failure, or perceptive impairment to prevent the data system from being aware of such disembodied creatures. I offer what I consider to be a common reason for hauntings in organizations: insulation of structural capital (related blogs: instrumentalism and institutional response). Imagine a highly fragmented body of interests pooling resources in order to achieve a common goal. For example, a number of nations might join forces in order to fight a war: maybe the war cannot be won if these forces are divided. If at some point the enemy disappears, only the fragmented body remains. All of the stakeholders that relied on the unity must then confront the possibility of disintegration. Similarly, many companies are empowered by diverse interests willing to invest in a common narrative of success. I suggest that the "American Dream" caused all sorts of social-structural developments over decades; the evaporation of that dream might have the effect of undermining the collective cohesive interests that made nationhood possible. As people, our identities are sometimes premised on a certain understanding of our role in and relationship to the world; upon our realization of "misplacement"- given that our sense of being cannot flip on a whim - it might be difficult to change not just our path but our perceived destination. Indeed, some people seem to turn to alcohol rather than acknowledge the loss. What options are available to ensure the cohesion (but not necessarily survival) of an entity or organization facing extinction? The answer is denial. An organization might simply refuse to believe in its ghosts.

In this blog, I have described how the absence of a critical component (e.g. citizens, clients, a shopkeeper, and transit riders) can cause a system or structure to experience distress - triggering the emergence of "ghosts." If the structure persists even on the absence of the component, the structure can become "haunted." For humans, this might mean that the structure itself is haunted by ghosts; or that the people occupying the structure are haunted. I have portrayed the persistence of the structure as pathological. The structure may have lost the ability to detect the absence of the component; or perhaps there is no willingness to accept its absence. Being alienated from its operational reality, the structure or organization might continue to its doom. Market termination is made more likely by the separation between data and fact - and perhaps more fundamentally between data and phenomena - between legs and torso. In the past, I have described this separation as a kind of decapitation. I wrote about "headless" data - incapable of retaining context due to methodological constraints. Suffice it to say that there is a severing of connection. The data that an organization accumulates might become increasingly disconnected from reality. Thus I draw a parallel between business termination from the market and the disassociation of data from the market - the latter contributing to the former.  This is a ghost story that plays out every day in organizations struggling not just with technology but threads of their existence in a world full of confusing and often threatening data.

Views: 286

Tags: alienation, capital, data, decapitation, detachment, disassociation, disembodied, ghosts, haunted, houses, More…insulation, participation, phantoms, public, spirits, structural, structures


You need to be a member of Data Science Central to add comments!

Join Data Science Central

Comment by Don Philip Faithful on October 25, 2015 at 6:52am

Thank you for your question - so soon after I posted the blog.  It depends on the setting.  In a production environment, there is usually a need to collect specific data and provide routine reporting.  There isn't an opportunity to go far beyond these parameters.  I therefore believe it is "normal" for organizations to structurally miss critical details.  In the case of pathology, the phenomenon of data-churning is evident:  this is when a compensation scheme is based on certain metrics of success; and so these metrics are exploited in order to support higher compensation levels.  I don't suggest a unchanging process that might be the same for all organizations.  Science dictates a systematic approach free of opinion and to the greatest extent possible social-construction and preconception.  To this end, I believe that a strategy of mass data assignment into organizational contextual arrays can provide inductive insights.  For example, assume that HR maintains absenteeism metrics.  These metrics can form the basis for mass data assignment thereby creating an open pool of associative influences on absenteeism.  Mass data assignment is a process that I developed to allow for floods of data to be attributed in a cohesive manner over different contextual arrays such as revenues, absenteeism, and product quality.  There is no presumption of knowledge, if that makes sense.  Developing methods of being sensitive to environmental articulation makes it less likely for preconceptions to limit to scope of substantive discourse.  My next blog on Conceptual Attribution will cover some details although I have already posted blogs on mass data assignment.

Comment by Kening Ren on October 25, 2015 at 3:44am
Thanks for sharing this blog and pointing out the "Ghost data". How to avoid it or fix it?

© 2019   Data Science Central ®   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service