Hard Problem of Consciousness - A research site

This site is maintained by Paul Hurren, a PhD physicist with a 20-year interest in the 'hard problem' of consciousness: What are the scientific and/or philosophical principles which explain our subjective experience? How is it that I (and I assume you) have a sense of self? Despite massive advances in neuroscience and related disciplines, these questions still remain unanswered.

The purpose of the site is to explain why consciousness is a real problem to be solved and to provide a brief introduction to the field. This is not an all-encompassing overview of consciousness studies, but rather signposts a small fraction of the great investigative work carried out worldwide from a scientific or philosophical viewpoint by researchers across several disciplines including neuroscience, philosophy, psychology and physics. The examples chosen for the Theories section are those which will be well known to researchers in the field and are likely to provoke discussion for years to come.

It is difficult to speculate at this time on which theoretical approach might be the nearest to a stepping stone to a broadly accepted theory of consciousness. However the site concludes with some Ideas that are hoped might contribute to the foundation of a successful theory.

The Problem

We experience the world around us through our senses. This all comes together in our brain, which somehow combines sensory information together with internal thoughts and feelings to create an overall experience of this present moment or 'now'. The areas in the brain which contribute to this experience, for example those parts of the brain responsible for processing sight, hearing or touch, and even specific regions responsible for shape or colour discrimination, are becoming better and better understood by neuroscience. The problem is, in essence, how this all comes together into a coherent whole and how it happens that there is an 'I' who is experiencing the world. How on earth do the 100 billion or so neurons in the brain (or alternative mechanism if there is one) do that?

Consciousness is our moment to moment experience of the world including our perception of “I” (the self). Given our brains are physical organs, how does this mental (seemingly non-physical) world come into existence?

The Hard Problem of consciousness, as compared with 'easier' problems, is a term famously coined by philosopher David Chalmers in the mid 1990s. Chalmers defines 'easy' problems of consciousness as being those which can be explained in terms of computational or neural mechanisms, whereas the 'hard' problem is one of experience, the experience of the colour red, the quality of depth in the visual field, the smell of a rose, the feeling of an emotion, and so on. The 'easy', arguably more mechanistic problems, seem to be more amenable to direct scientific investigation.

Not everyone in the field agrees there is a hard problem. In particular Daniel Dennett in his 1991 book Consciousness Explained argues that our sense of self is in fact a conjuring trick performed by our brains, it is an illusion which can be explained in physical (computational) terms. Or an often heard argument in the field generally is that once we have a better understanding of how the brain functions, a theory of consciousness will become more and more within our grasp: Once we have solved the 'easier' problems then a solution to the hard problem will be in sight.

But the balance of evidence supports the view that the problem of consciousness is in a harder league than other problems in neuroscience, in a league of its own. Despite decades of research by thousands of investigators, we still lack a clear and widely accepted explanation for how the conscious mind, a world of experiences, thoughts and feelings, comes into existence. Our knowledge of the brain and its various functions continues to grow rapidly - if a theory of consciousness were to 'fall out' of a better understanding of neural processes we ought to be getting reasonably close by now. Some prerequisites for consciousness are well understood by neuroscience though, including what if damaged in the brain would lead to consciousness being taken away.

So although I am conscious, or more correctly I have an experience right now (it appears to me) that there is an "I" who is conscious, the explanation for this is as yet a complete mystery to the scientific world. Although there have been many theoretical attempts to explain consciousness, we do not know yet whether any one of them is even just laying some very basic groundwork for a credible theory. But one or two might be.

This is a problem to be solved for its own sake - pure research. But any real solution will likely be of major benefit in practice to disciplines such as psychology and psychiatry. A working 'model' of consciousness, of what is going on to give us our experience of the world, will provide a step-change in our understanding of how the mind works and give improved insight into human psychology and hopefully into mental illness too. On a wider philosophical front it might also introduce a new perspective on our relationship with Nature, just as have some key fundamental scientific discoveries done in the past.

Mind and Brain

There is still only a limited understanding of the relationship between mind and brain. By mind I refer here to the overall manifestation of thoughts and feelings, both conscious and unconscious, that take place within the brain. A very crude analogy put forward by some is the brain is the 'hardware' whereas the mind is the 'software'. This analogy can be useful to illustrate a key difference between the two but should not be taken too literally. The conscious mind as compared with the unconscious part is that subset of mental processing which directly relates to the contents of consciousness, our experience of the here and now, our sense of self. The term 'processing' is used here very generally because (referring back to the hardware-software analogy) there are a number of arguments (e.g. Penrose) to suggest that consciousness is not entirely computational, something other than pure computation by the brain needs to be thrown into the mix. And it should be emphasized that the underlying processes which give rise to consciousness will themselves (by their very nature) be unconscious or 'preconscious' in nature.

Theories speculating on the relationship between mind and brain broadly fall into one of 3 categories: (1) Mind = Brain, otherwise known as Identity Theories (see for example Thomas Polger) where mental states such as sensations are considered to be identical to physical states, (2) The mind is a separate substance from the physical brain (dualism) stemming from Rene Descartes ("Cogito ergo sum: I think therefore I am") who believed the pineal gland in the brain being "the seat of the soul" was the connection between the two, but more commonly represented today by panpsychism where consciousness is considered to be a fundamental property or aspect of nature, and (3) theories where mental processes are emergent from physical processes in such a way that there are neural correlates of conscious experiences but not necessarily a direct relationship between the two.

Philosophical approaches

The mind-brain relationship can be considered and categorized in purely philosophical terms. Physicalism which has many variants maintains that there is nothing more than the brain/body and its interaction with the world - the mental world is an emergent phenomenon from physical processes. The challenge here is how mental processes do indeed emerge from the physical. As yet, while there are several examples of weak emergence where something (for example sound waves) arises from the collective behaviour of constituent parts (in this case air molecules), there is yet no proven case of strong emergence, where the emergent property cannot be derived from the constituent parts and the interactions between them. Producing a mental world from the physical world through emergence would intuitively seem to require emergence of the strong variety, essentially an extension of our understanding of the physical world, but consciousness through weak emergence is still a possibility, we just do not know. Chalmers has reflected in strong and weak emergence in the context of mind-brain.

Dualism as described above treats consciousness as something separate from the physical world. While this appears to eliminate the problem of how something mental arises from a physical process of some sort (e.g. neural firings), there are many challenges, starting with how a separate mental world or substance would connect with the physical one. There is no question whatsoever (from neuroscience and otherwise) that the brain has something to do with consciousness, and depending on one's scientific and philosophical outlook it will have a greater or lesser causative influence. So for dualists, explaining the interaction of a non-physical substance or entity with a physical organ is paramount. With panpsychism, one of the challenges (if consciousness is fundamental to the world) is how the various elements of conscious experience (often referred to as 'qualia'), such as one's experience of the colour red or of a certain musical note, combine to make a unified whole. This is known as the Combination Problem.

In contrast to dualism, Idealism holds that all the material world arises from conscious activity, that is, reality is fundamentally mental. Like dualism or materialism idealism has several flavours. A useful overview from a mind-body perspective is provided by Chalmers. While Idealism has had a big impact on philosophy it is arguably not a front runner as a basis for the explanation of conscious experience; at least it would require a massive upheaval to the mainstream scientific world view.

A fourth philosophical approach known as Neutral Monism maintains that the world is neither mental nor physical. Instead the mental and physical worlds are two aspects of an underlying neutral reality. This approach has appeal in that it is generalist, acknowledging that both mental and physical are different 'views' of the same reality, but the challenge is to explain the relationship between the mental and neutral entities, and likewise between the physical and neutral entities, in effect to better understand what those neutral entities might be.

Theories of note

There is so much good work on consciousness out there, with contributions from many disciplines. This is to summarize a few which stand out from my own personal (knowledge and) perspective, which have either helped the field take a step forward (and are widely acknowledged to have done so) or have provided a provocative argument which might help an interested reader better understand their own world view.

Francis Crick focused the last 25 years of his life on research into the brain. Crick was surprised that so little attention had been paid to consciousness since William James' The Principles of Psychology about 100 years previously. After extensive reading across neuroscience, psychology and psychology, Crick became convinced that the answer to consciousness lay at the neural level. Along with his research colleague Christof Koch he developed a framework for consciousness comprising "a coherent scheme for explaining the neural correlates of (visual) consciousness in terms of competing cellular assemblies".

The framework focuses on time periods of typically a few hundred milliseconds. So called 'zombie modes' in the cortex respond rapidly and unconsciously to sensory inputs with a flow of information that is likely to be in a single direction, a kind of feed-forward wave of neural activity ('net-wave'). Conscious modes are slower acting and might comprise a flow of information in both directions like a standing net-wave. It is suggested coalitions of neurons form where each neuron supports the activities of other members; these coalitions compete and the winning coalition (where activity goes beyond a certain threshold) achieves conscious status. Coalitions at the front of the brain, which interact with coalitions at the back, may reflect feelings. The framework speculates that for visual conscious awareness, perception occurs as a series of snapshots, each snapshot, for example suggesting motion of an observed object, is represented by a constant rate of firing of a given group of neurons. Binding, the bringing together of different aspects of an object or event (e.g. shape and colour) might be achieved through the relevant neurons being members of the same coalition, with coalitions distributed over the back and front of the brain.

The above framework addresses the neural correlates of consciousness (NCC) rather than to approach the hard problem of consciousness "head on"; Crick and Koch believed that a better understanding of the NCC might shed light on the solution to the hard problem.

Bernard Baars is the originator of the Global Workspace Theory which he proposed in the 1980s, a theory which like Crick and Koch takes consciousness to have a biological basis and is arguably the most influential theory of consciousness to date. While the brain has powerful parallel processing capabilities the contents of one's immediate 'stream of consciousness' are limited in number (1 to a maximum of 4 items at any one time, and up to 7 or so items in working memory). This contrast between vast unconscious parallel processing and limited conscious serial processing might provide a clue. Baars suggests that consciousness acts as a gateway to unconscious processes and gives a number of different functions access to each other - consciousness is required to integrate two or more pieces of information. According to Baars, consciousness involves an interaction between the 'observing self' and objects of perception (which include an observed self), within a global workspace which is in effect a spotlight or focus of neural interactions across wide areas of the brain. As a metaphor, in the 'working theatre' which is the global workspace, consciousness acts as a bright spot on the stage (but note Baars' theatre is not a Cartesian one where everything comes together at a single point in the brain). Baars hypothesizes most of the interactions directly correlated with conscious experience occur between the cortex and thalamus, with different areas of the cortex becoming contributors to the global workspace over time. The theory is broadly consistent with experimental findings of conscious experience correlated with broadcasts of neural activity.

Global Workspace Theory, as Baars himself would admit, cannot be regarded as a complete theory of consciousness. While it provides a framework for further investigation it does not solve the hard problem as such, for example to explain how our feeling/experience of 'self' arises. However the theory might provide an important piece of that jigsaw puzzle which is the full solution.

The neuroscientist and psychiatrist Giulio Tononi has led on the development of Integrated Information Theory (IIT) which focuses on the necessary conditions for consciousness from a neural perspective. IIT starts from consciousness itself (which it takes as certain) by considering its essential properties:

• Intrinsic - Consciousness exists
• Composition - It is structured
• Information - It is specific
• Integration - It is unified
• Exclusion - It is finite in content

IIT identifies a set of postulates for the properties of physical systems which can account for these properties and develops a mathematical model around this. The kind of physical systems under consideration here have elements in a state, such as neurons and logic gates, with each element having two or more possible states. Key properties (postulates) are that the system must have cause-effect power (it must be able to make a difference to something or be made a difference to, including to itself); subsets of the system must have cause-effect power on each other; it needs to be specific (the particular way it is); it must be unified and irreducible (every part of the system must be able to affect every other part); and the cause-effect structure must be definite, no less and no more. IIT then proposes a central identity: An experience is a maximally irreducible conceptual structure (as specified by a particular set of elements in a state). IIT defines Phi as Integrated Information, a measure of the intrinsic irreducibility of the system. The maximum irreducibility, Phi Max, gives an indication of the level to which the experience exists, essentially whether any system of elements in a certain state might be conscious. An experience is a form in cause-effect space, and the quantity of the experience (how much consciousness there is) is measured by Phi Max.

Tononi points out that "assessing this identity systematically is difficult, mathematically, computationally, and experimentally", but has carried out a number of simulations using the theory with results consistent with the loss and recovery of consciousness due to dreamless sleep and waking, and in relation to certain brain lesions. But to fully test the theory experimentally he points out the mathematics will need extending and experiementally it will be necessary to establish "whether changes in the physical substrate of our own consciousness are related to changes in experience".

While IIT makes a comprehensive and credible attempt to define under what conditions a physical system might be conscious, it is ultimately an identity theory, and falls short of explaining (yet at least) how it is that a physical system can have subjective experience and a sense of self.

Daniel Dennett a philosopher of mind, first published his multiple drafts model in a 1991 book 'Consciousness Explained'. The model is proposed as an antidote to what Dennett terms Cartesian Materialism, the concept that after some stages of unconscious processing, everything comes together at a single point in the brain (analogous to Descartes' pineal gland) and there is some significant (and temporal) transition between unconscious processing and conscious awareness.

With the multiple drafts model, a conscious state is spread out over space and time. When information from the senses is processed (unconsciously) the brain makes hundreds of thousands of 'micro judgments' to analyse what is going on. There is a great amount of two-way activity between higher and lower brain areas where a number of accounts (drafts) are developed and compete for domination in the brain ("fame in the brain" or attention). What we think of as the stream of consciousness is actually (in Dennett's words) an "edited history which is only retrospectively available".

The model challenges those theories (I doubt any physicalist theories cited on this website will be open to this challenge though) which seem to invoke some 'inner observer', and is certainly helpful in that respect. But Dennett does not think there is a hard problem, he feels that proponents of the hard problem are re-making the 'vitalism' mistake (that life depends on some non-physical force). But that is not how the hard problem is framed here or by most in the field.

Theoretical physicist Roger Penrose and anaesthetist and psychologist Stuart Hameroff have a theory known as Orchestrated Objective Reduction (Orch OR). This proposes that the causal factors behind consciousness lie at a level below the neuronal one, that is at the level of microtubules within neurons, and extend down to the quantum level. Microtubules are cylindrical polymers, approximately 1 billion per neuron, with diameter ~25 nanometres and length from a few hundred nm to metres. The theory proposes consciousness comprises a series of discrete events, each a moment of Objective Reduction of a quantum state within neuronal microtubules (OR considers the quantum wave function to be physically real and the collapse of the wave function, essentially the bridge between quantum and classical worlds, does not require observers to have a special role). For consciousness to arise, these OR events need to be coordinated or 'orchestrated' and over a sufficiently long timescale for coherence. The theory touches on multiple areas of science including (through OR) quantum gravity and suggests consciousness is intrinsically connected with fine-scale space-time geometry.

While Orch OR doesn't appear to solve the hard problem (yet) as such, it is clearly a bold and cross-cutting theory with quite a few moving parts that has generated a number of testable predictions. It is sure to provoke discussion for years to come.

Neuroscientist Antonio Damasio has published some ground breaking books including Descartes Error which lay the groundwork and The Feeling of What Happens which sets out a comprehensive theory of consciousness. For Damasio, feeling and consciousness are closely intertwined. Feelings such as well-being, hunger, desire and fatigue are fundamental and homeostatic (in that they convey information about the state of life within the living organism and support life processes to continue). Although feelings (and non-neural biological processes in general) are seldom central to conversations on consciousness, to Damasio minds are constructed from a cooperation of non-neural as well as neural events.

How does feeling intervene in the process of consciousness? The answer lies in the ability of the brain to represent entities and events, both internal and external. As part of the homeostatic processes for survival regulated by the brain's core, brain stem and hypothalamus, the brain maintains a neural map of the whole organism so that it can monitor for any changes which occur. Damasio suggests that the brain creates a second-order representation using structures which map both the organism and external objects, and this representation "presents within the mental process the information that the organism is the owner of the mental process" thus creating "the sense of self in the act of knowing" (and hence a first person perspective) which "emerges as a special kind of feeling - the feeling of what happens in an organism caught in the act of interacting with an object." Damasio's approach to this has quite a lot in common with that of Nicholas Humphrey who also sees feeling as being an important contributor to consciousness.

Damasio's emphasis on homeostasis and feeling and the link to consciousness is echoed by some other researchers, in particular Mark Holms. In his paper The Hard Problem of Consciousness and the Free Energy Principle Holms sets out his thesis, similar to Damasio's, that the elemental form of consciousness is affect (feeling) and its physiological mechanism is located in the upper brainstem. He cites significant evidence that consciousness exists in the absence of a cerebral cortex and can be obliterated by small lesions of the brainstem core. Children born without a cerebral cortex (as a result of hydranencephaly) show emotional reactions to events in the environment. To Holms, "One surely must conclude that it does feel like something to be these children", we need to accept that they display basic emotions. The cortex on the other hand, rather than being the seat of consciousness as many researchers believe, is likely to be the equivalent of random access memory and while important for higher order cognitive functions is not essential for core consciousness itself.

From a theoretical perspective Holms proposes free energy minimization as the basic function of homeostasis and as the mainspring for consciousness itself. Minds emerge as a result of self-organizing systems monitoring their own states. While the theory puts forward some interesting concepts, it doesn't seem to convincingly explain how subjective experience arises from neural activity. Holms' own philosophical stance is one of dual-aspect monism where brain processes associated with consciousness, and consciousness itself (the first person perspective), are regarded as two observational perspectives of the same thing. But such a dual-aspect approach appears to neatly sidestep the hard problem.

From the experimental evidence cited by Damasio and also by Holms it is clear that consciousness is a multi-level phenomenon. Core consciousness which depends on the brainstem is associated with basic feeling and extended consciousness involving the cortex is associated with higher order awareness / cognitive functions. In due course the introduction of feeling into the equation might be recognised as an important stepping stone to solving the hard problem. And the linkage of feeling and consciousness would put the evolutionary 'rationale' for consciousness in clear perspective.

Ideas

For the remainder of this site an attempt will be made to lay down some building blocks towards solving the hard problem. The approach will be a physicalist one, but recognising that within this magnificent (physical) universe we inhabit there is a mental, subjective world which seems very real. Can the mental and physical be put into a common framework, and can that framework be applied to give insights into how subjective experience arises? We shall see!

A process or a thing

Is consciousness created/enabled by a process, or is it a thing in itself?

The general physicalist stance is consciousness arises from a process of some sort (yet to be understood!) which is associated with the brain and consciousness would not exist if that process did not occur. This is consistent with clear neuroscientific evidence where damage to parts of the brain causes impaired mental function or (more relevant to consciousness) complete loss of consciousness itself.

Our experience of the present moment appears to have a joined-up 'oneness' about it - what we perceive and feel right now is an integrated whole. But over times longer than a few hundred milliseconds we are constantly experiencing new things. Even when the visual scene is relatively constant, our attention will change. So the contents of consciousness as the snapshot of what we are aware of at any given time are momentarily constant, but those contents are continually changing. Driving this change must be a process of some sort.

We know the brain is essential for consciousness and much is understood at a neuroscientific level of how the brain processes sensory information, so at the very least there is processing of sensory information going on. Also however, something is happening to 'bring things altogether', and that something from a physicalist point of view must be a process.

With panpsychism, which states that consciousness is a fundamental property of nature just like mass or electric charge, consciousness through its fundamental nature is a 'thing' (a property) rather than a process. Taking the electric charge analogy, for each electron its charge is always present. But if consciousness were a fundamental property analogous to charge, how does consciousness switch off at night when we go into dreamless sleep? If there is an answer to this from the perspective of panpsychism it must be something to do with how those fundamental units of consciousness combine, so we are back to the Combination Problem. Or perhaps consciousness is analogous to electric current (moving electric charge) which can be switched off. Either way, even for panpsychism, consciousness will depend on some kind of process.

This needs to be a starting point for any attempt to solve the hard problem. Note that the contents of consciousness are likely to be 'things' (i.e. objects). We must distinguish the contents from the process which enables those contents to be experienced, both of course are important.

The arena of consciousness

Any investigation into the underlying principles behind subjective experience should have its 'arena' defined. That is, assuming there is a process of some sort that gives rise to consciousness, what over-arching factors play a role in that process and need to be taken into consideration?

We consider 3 key macroscopic factors, the brain, the body and the local environment that the brain/body interacts with.

That the brain is important is self evident - whatever the underlying explanation for subjective experience, the brain has an important part to play! With consciousness as a process of some sort, the brain is the 'engine' of that process.

Secondly, while on a physical level the body sustains the brain, the brain/body system according to neuroscience plays an important role in the generation and experience of feelings/emotions. And the body, through the senses, is the brain's 'interface' with the outside world.

Thirdly, without interaction with an outside world there would be no stimulating environment in which the brain could develop its higher abilities, for example in learning through trial and error. This learning and development through interaction with the world is applicable both to the growth and maturity of a single conscious being as well as to the longer-term evolution of a species of conscious beings (e.g. the evolving intellectual abilities of humankind).

Note that while the external world is key for the development of higher forms of consciousness (such as reasoning abilities) the role of the external world as a driver for core or primitive consciousness is debatable: If core consciousness is intertwined with homeostasis as per Damasio and Holms, to sustain core consciousness interaction with the outside world might not be necessary.

So the brain, body and environment are all important.

Consciousness and physical reality

Another aspect in relation to placing consciousness in a wider context - it is a phenomenon for which there is no straightforward explanation within the bounds of current scientific understanding. To get a proper handle on consciousness it might be necessary to reconsider how we describe the physical world. There might be a different way of describing the world in scientific terms, or a different perspective, such that consciousness becomes a natural 'consequence' of that description/perspective.

Any attempt to account for subjective experience from a physicalist viewpoint will invoke a mechanism or mechanisms of some sort. But given the mental world seems to be 'extra' to the physical world and its explanation does not currently fall within the realm of physics, more than just a new mechanism is required. Something fundamental needs to be added, otherwise it would be like trying to understand three dimensional space (say) from the perspective of two dimensions. So bringing consciousness into the framework which describes the physical world will require one or more new physical principles too.

Visual perception: Where objects as perceived are located

With visual perception, the generally accepted model is one of an external stimulus being received (light rays to the retina) and through a number of steps of neural processing (including colour, edge and shape discrimination, motion detection, object recognition etc) a visual scene is constructed by the brain as an internal 'representation'. Here any given representation will comprise the neural patterns which correlate with (represent in neural terms) a specific conscious experience, in this case the subject's experience of the visual scene at a given moment. The term representation, while well established in the field, needs to be treated with caution because it should not be interpreted to imply the visual scene is represented to something (such as an 'inner observer'). Rather it describes an encoding by the brain where there is a mapping of some sort between the contents of consciousness and neural activity.

But in experiencing an object within the visual scene, where is the perceived/experienced object located? It is widely assumed that the perceived object (as distinct from the actual, physical object) will be located in the mind/brain of the observer. This is no doubt the popular/mainstream view for a number of reasons, including: (1) Consciousness is 'mental' not physical, so surely all experience / contents of consciousness are 'inside the head'; (2) It is assumed that the brain constructs internal 'images' of objects, therefore conscious experience would involve interacting with images rather than directly with the physical objects themselves; (3) Phenomena such as visual illusions or hallucinations seem to suggest that the brain is constructing an internal three dimensional virtual reality rather than directly interacting with the external world.

It could be argued though that the above assumptions are on shaky ground. While whatever process which gives rise to consciousness takes place in the brain, (at least) some of the contents of consciousness could in principle be external. If the brain constructs representations of objects which make up an internal 3D world, how is this internal world kept (rapidly) in sync with the real external world such that one is never aware of there being an 'intermediary' world in one's head? (e.g. when moving the head, one's perspective of the world appears to adjust instantaneously). And with visual illusions might there be an alternative explanation for why conscious experience is not a direct 'copy' of reality, for example the brain/senses are executing a transformation on input signals received?

Max Velmans in his 1990 paper Consciousness, Brain and the Physical World and developed further in his book Understanding Consciousness argues that the visual experience of an object is the object that is seen out in the world. The perceived object is out there, as one sees it, not 'in the mind or brain'. Velmans develops what he calls the Reflexive model of perception whereby in the interaction of an observer with an object there is a 'reflexive' preconscious interaction of observer with observed and the experience is of the object "projected to the judged location of the initiating cause". While within the brain there are neural correlates of consciousness and a neural representation of the object as perceived, the actual experience is one of an object in the world out there in three dimensional space, an object as seen. Velmans applies similar reasoning to other forms of experience, for example a pain in the finger, while mediated by the brain, is experienced 'in the finger' rather than in the mind or brain itself. With Velmans' reflexive model, physical objects as perceived are part of the contents of consciousness; an object as experienced is the same as the experience of an object. While the experience has neural correlates, the experience is not the same as the neural correlates. Velmans introduces a dual aspect theory - a complementarity principle, where first and third person perspectives (the experience itself on the one hand and the neural correlates on the other) are complementary and mutually irreducible, they are two complementary accounts of what is going on.

Riccardo Manzotti in his book The Spread Mind goes further. While Manzotti states "Consciousness is where and when the physical objects that one experience take place.....Our experience is the world we live in" which sounds on the surface a lot like Velmans, to Manzotti there is no internal representation of the object in addition to the experience and no 'projection' of the object into three dimensional space - there is just the real physical object as the contents of consciousness. The brain/body does play a role though, which is to offer "the causal circumstances that allow the objects that one experiences to exist" (i.e. through senses/perception). Manzotti's proposal is a purely physicalist one: Our experience of an object is the object itself, perception is identity with the physical object, and subjectivity is no longer needed. It is a simple theory which, while conceptually might not be easy to grasp especially for those from a more 'mainstream' psychology or philosophy of mind background, challenges conventional wisdom. Novel and provocative ideas such as this might help propel things further forward.

Reflecting on the above subject matter and on whether physical objects are experienced as external projections (Velmans) or actual physical objects (Manzotti), if we consider the interaction of the brain/body with the environment before a new visual scene is consciously perceived, light rays from an object hit the retina and a series of information processing steps take place within the brain including colour, edge and shape discrimination, motion detection and object recognition. While there is a conscious aspect to object recognition, especially combined with attention, some elements at least are carried out preconsciously, see for example Johannes Fahrenfort's PhD thesis on Conscious and Unconscious Vision. If this is the case then there is initially an unconscious interaction between the brain/body and a physical object. So when later the object becomes part of the contents of consciousness, it would not be unreasonable to suggest that our conscious perception is of the physical object out there in the world. In other words, if there is a direct connection/interaction between brain/body and the environment at an unconscious level, why shouldn't this direct connection extend to the conscious level, there is no need for any internal 'imagery' in the brain and real objects as perceived/interpreted by the senses will indeed be part of the contents of consciousness.

To reiterate, from the point of view of basic physics, the (physical) brain body is engaging with a physical object through the senses. Just for one moment let's imagine an image of the object is (replicated by the brain to be) inside the head. Observable changes to the physical object in the external world would be reflected in changes to the image of the object in the head, for example a real pendulum set in motion which is perceived will result in an internal image of a pendulum set in motion, albeit with some local transformations (e.g. colour). There will need a synchronisation process of some sort, let's call it 'Process S', with a second, interconnected process in the brain, 'Process C' giving rise to conscious experience. So there is 'Process S' (which creates the image) and 'Process C', but the end-to-end process is one where the physical brain body engages with a physical object and conscious experience (of that object) somehow arises. What then is the purpose of that internal image, why is it needed? Why can't we combine 'Process S' and 'Process C' into one? In truth we are talking about a single overall process of conscious perception here!

The concept of perceived objects being inside the head is a dualist way of thinking, one which invokes separate mental and physical worlds. This needs to be challenged.

The reference 'as interpreted by the senses' is important because there is a transformation going on by the senses and brain, a transformation from the attributes of the physical object to the attributes as perceived. This will be discussed in more detail later on.

Visual perception: Experiencing an object

Our experience of a physical object is not only of the object itself. The reasons are as follows:

• There is the feeling/perception of a 'self' (an 'I') experiencing the object, a subject which sees the object from a certain perspective and which is affected by the object. You are looking through your eyes at an object and are aware of your own body within a 3D world. The self seems very real, in fact it is probably the most (apparently) real thing each of us know!
• It includes a rapid internal (mainly unconscious) process of categorization ("this is a cat") and association ("the cat from next door") which gives rise to the experience a perceived object with meaning. At least some of the categorizations and associations made will be internal to the mind/brain rather than being intrinsic properties of the object itself. For example, "my neighbour's cat" as perceived by me (an association of 'cat' with 'neighbour') might just be "a cat" to someone from another town perceiving the same animal. (However both of us will still see the object as a cat so being cat-like can arguably be said to be an intrinsic and objective property of the cat itself)
• The object might also invoke internal (to the brain/body) feelings: "I really like that cat"

So a straightforward description of perception could be as follows:

1. Physical objects exist in their own right
2. We perceive physical objects as the objects themselves, not projections or representations
3. Perception and object recognition start unconsciously and are triggered by the object as cause
4. The subject is affected by the object
5. There is the feeling/experience of being a subjective observer of the object, and
6. There is a rapid and mainly unconscious process of categorization and association which attaches meaning to the object

Therefore with reference to the preceding section, while Velmans and Manzotti both (separately and for different reasons) propose that an object as experienced is the same as the experience of an object, there is also an internal aspect to experience which includes feeling, categorization and association and/or recall from memory.

Mental and physical

Is there a way to put 'mental' and 'physical' into a common framework at a fundamental level in a way which is consistent with the mainstream scientific world view? The approach here is to start from a physicalist viewpoint but accepting there is subjective experience which is real to the conscious being and a 'mental' and 'physical' perspective to the world. It is suggested here, as we shall see, that there is significant overlap between mental and physical in relation to a conscious being's engagement with and experience of a physical object.

Note that the word attribute will be used here in the context of objects and their attributes. Attribute is broadly synonymous with 'property' but is used here instead of property to refer to a specific property an object has, for example the petal of a red rose has the property of redness but an attribute of being a specific shade of red.

As a starting point it might be helpful to remind ourselves what is actually physical about a physical object!

A woodburner - this everyday object to most observers is purely physical. But what aspects of the object are actually physical?

I see in front of me an object with external attributes. 'External' refers to those attributes which can be readily observed/experienced in practice and which are macroscopic, such as mass, size, shape, texture and colour. The object will also have internal attributes such as its atomic/molecular structure and so on, but we are concerned here with those external attributes which an observer can experience. This particular object also creates sound waves as the wood burns.

The object with mass, size, shape, texture, colour and sound is the object I perceive, but what is the nature of the actual object?

• On mass, size and shape, these it is suggested are intrinsic to the object itself and are a consequence of the substances which make up the object and how they are combined and formed. Saying that, none would still be directly observable without the senses and without a brain to create the perception of (being the observer of) an object situated within a three dimensional world.
• Texture is in effect fine-grained shape, also intrinsic to the object.
• With colour, the object reflects light of certain frequencies but were it not for an observer, the object would be colourless in the way we think of colour. However the object still has an intrinsic attribute, its surface spectral reflectance which has a 1:1 mapping with colour. The reflectance attribute could therefore be described as 'colour' in the sense that a specific reflectance attribute will give rise to a specific colour when perceived by a given observer (subject to any contextual transformations by the brain, for example in the case of colour-related optical illusions).
• For sound, the object (burning wood in this case) creates air vibrations in the audible frequency range (sound waves). While no sound as such would be heard without an observer present, sound waves are nevertheless emitted as a physical consequence of the interaction of the object with the surrounding air.

So while the external attributes of the object are intrinsic to it, the way those attributes are experienced depends on how sensory information is processed by the brain/body of the observer: There is an intimate relationship between the object as perceived and the observer. Furthermore the attributes of an object as perceived are not completely separate from the attributes of the physical object, for example there is a direct mapping between colour and surface spectral reflectance.

Considering the mapping between perceived and actual attributes, and the process of perception as outlined in the preceding sub-section, it is clear that the brain, body and objects in the environment are working together as an integrated system. This contrasts with what seems to be a popular concept of an internal 'mental world', a so-called "movie in the head", which is somewhat detached or isolated from the physical world. It is proposed here that there is no such isolation between the mental and the physical. If the brain creates a 3D model of the world, that model will be tightly coupled with physical objects in the local environment, those objects which make up the current visual scene and contribute to the experience of 'now'.

The external property/attribute of form (shape, and through surface spectral reflectance, colour) is physical but is also abstract in nature. Form can be reproduced, just as the image of Sunflowers below is a copy of a physical object and that copy as well as the original object has abstract content, the art/image itself. This abstractness is part of the physical world.

When I perceive a version of 'Sunflowers' by Vincent van Gogh I am struck by what seems to me to be such energy and emotion in the painting.

According to what has been outlined in this section, the interaction between the object and my brain/body initially takes place at an unconscious level and at some point the object (in this case the painting) becomes part of the contents of my consciousness. Feelings and thoughts become associated with the object as part of the process of perception. In general terms it could be said we are living in a world of objects, and the contents of consciousness at any time comprise objects as perceived (or imagined) from the perspective of the brain/body.

But on further reflection it is really the form of an object as perceived (its two or three dimensional shape and orientation) combined with its size and colour(s) that comes into consciousness. That is, the most impactful (to the perceiver) external attributes of the object come into consciousness.

Form

We define 'Form' (with an uppercase F) to be a composite attribute ('composite form') comprising those attributes that are capable of being perceived. This will be principally size, form (shape) and colour from the point of view of ordinary day to day interaction with objects, especially those experienced from a distance. Mathematically we can write F = {s, f, c} where F is composite form (the set of perceivable attributes of an object) and s, f and c are the individual attributes of size, form (shape) and colour. The curly braces { } designate a set. We can include other attributes of the object in this type of framework.

The following hypothesis is put forward:

• Thoughts seem abstract; the mental world seems abstract
• Physical objects have Form which although real is also abstract. For example a real tennis ball is green (say) and has a spherical shape but a green sphere itself is also an abstract object
• The (conscious) mental world is a world of (abstract) Form. The physical world is also a world of Form (as related to objects)
• There is an intimate connection between mental and physical, through Form

The above implies an equivalence between the mental world as experienced and the world of (physical) form. The abstract world of objects (the Form of physical objects) and what we consider to be the mental world are made of the same stuff - there is no physical-mental divide! If this is the case, in a conscious being's interaction with the world, the contents of consciousness are (the Forms of) actual physical objects.

From a physical (physics) point of view:

Form as defined here is the set of attributes of an object which are capable of being perceived, those macroscopic external attributes of an object which a conscious being (given an appropriate position and orientation relative to the object) can become aware of. This set of attributes might in principle be broader than those attributes capable of observation by humans, but that could be considered to be 'fine tuning' at this point in an analysis which is focused on the fundamental principles involved. The concept of form as fundamental is not new - it goes back to Aristotle.

If Form can indeed be regarded as a fundamental aspect of the physical world, but also Form is abstract in nature, there is a link between mental (abstract) and physical, through Form. And to help explain subjective experience in physical terms we need a link such as this which places mental and physical into a common framework.

Time

It is well established that the brain has specialized regions for processing different visually perceived attributes such as shape, colour and motion. And these attributes are processed at different speeds: First colour, then shape/orientation, then motion (Zeki). Furthermore while consciousness can be described as the subjective experience of 'now', that 'now' is not instantaneous, rather it can be regarded as a brief rolling (continually moving) time window. When we experience a visual scene, that visual scene is not a single snapshot in time but is quasi-continual as evidenced by (say) the ability to see and follow a moving object. Based on visual object recognition speed being typically 150-200ms, perhaps the minimum time window analogous to 'now' which provides continuity to the conscious being's experience of the world will be of the order of 0.5 seconds or so. And object recognition is only one of several essential supporting processes (depth perception being another) which require non-zero processing times.

So not only does the brain integrate various information sources (inputs from the senses etc) to produce the perception of a joined up whole, it does so over a finite time interval, an interval which because of various processing times needs to be non-zero for coherent subjective experience to arise.

Real and 'virtual' worlds

Let us consider what Form as fundamental might mean from the point of view of a conscious being. A popular (and possibly widely accepted) hypothesis is that the brain creates a kind of 'virtual reality' - the world as experienced (the phenomenal world) is a virtual one - the brain and its mental processes never directly engage with the actual world, rather the brain/body interaction with the local environment is mediated by this 3D virtual model. While there is good evidence that the brain does create a virtual model, for example in the way a conscious being has the experience of being in 3D space even though the information received through the senses is two-dimensional, we need to question what aspects are truly 'virtual'.

With the hypothesis advanced here, Form being intrinsic to the physical world and the mental world being the world of Form, when the brain/body is interacting with objects in the physical world, those objects which enter into conscious experience are real objects. It is the felt or perceived self together with the brain/body's perspective on the objects it is interacting with which are the 'virtual reality' aspects. The process which gives rise to conscious experience is enabling the brain/body to participate in, and directly interact with, an abstract (and physical) world of objects, a world of form.

Note that 'objects in the external world' will include the body itself.

Consider everyday perception. There is close engagement, through brain/body interaction with its local environment, between the physical world of material objects and the subjective world created by the brain. When physical changes (especially macroscopic changes which enter quickly into awareness) occur, the world as perceived will change too. A ball is thrown to me; the physical ball moves towards my physical body; I notice the ball; as the distance between the ball and my body decreases, my perception is one of the ball approaching me. This might sound trivial and obvious, but 'mental' aspects of perception are closely tied to the physical world, to the set of physical objects which (from the brain/body's viewpoint) comprise the visual scene. While a conscious being has a certain degree of control over what it pays attention to, much of its 'virtual reality' experience is driven by, and closely linked with, physical objects/events. In fact a good proportion of its conscious experience, that part of its experience which arises from the senses, will be driven by physical events.

To recap, we have the physical world comprising the world of objects, and an abstract world comprising the world of the Form of those objects. Just as the Form of a single object is intimately bound with the physical object itself, so the abstract world of Form is intimately bound with the physical world. As 'the thing which encapsulates matter' as well as 'what abstract is made of', Form is the 'interface', 'link' or 'common element' between physical and mental.

While the subjective, mental world including the self are created in some way by a process within the brain, this does not mean it is not real. While not directly observable to a third party in the physical world, it is real from a mental, subjective perspective. And the outcomes of that mental world (what the conscious being actually does, how they behave as a result of having subjective experience) are very observable.

Interactions between objects

The brain/body of a conscious being is an object, albeit a sophisticated one. It interacts with its environment physically, as in the force in the hand/arm when a baseball is caught, and mentally as in perception. In the previous section it was emphasized that the mental, phenomenal world is closely bound with the physical world. Furthermore in brain-body interaction itself, mental and physical are tightly intertwined (e.g. feelings and emotions).

Just as two billiard balls colliding on a frictionless surface have resulting trajectories and velocities determined by the laws of conservation of energy and conservation of momentum, taking the physicalist stance, the experience a conscious being has of a physical object (its perception of it) must be governed by physical law. The object ("ball A") has 'impact' on the conscious being ("ball B") in that, along the lines of Damasio, "the feeling of what happens in an organism caught in the act of interacting with an object", it affects the conscious being. 'Impact' in this case can be described in two complementary ways, firstly (subjective description) as the perception the conscious being has of the object and secondly (objective description) as a change of some sort in neural patterns in the brain. That change in neural patterns, or more correctly (because conscious experience is not instantaneous, it needs a certain elapsed time) a particular set of changes of neural patterns over a short time interval (say 0.5s), will be correlated or mapped with the conscious being's subjective 'view' of the object and feeling of self.

A footnote to the above: Damasio in his book Self Comes to Mind states "I use the terms image, map and neural pattern almost interchangeably". But are neural pattern and image really equivalent? Yes perhaps if one regards 'image' as 'a grid of pixels' one can postulate that the image of an object is equivalent to a set of neural patterns (or a 'map') within the brain. But if one regards 'image' as what is seen subjectively, that is, form, there cannot be an equivalence between image and neural patterns, only a correlation, Why not equivalence? Because there is an as-yet-unexplained transformation going on between what is captured at the 'map' level and what is experienced as form!

A conscious being's subjective view of an object is its view of the body in relation to the object. When conscious, we experience the world with our own body as one of the objects in the visual scene, we observe our own bodies in relation to objects in the environment. So what is created by the brain is not simply a view of the object itself, it is a view of the body in relation to that object within the overall visual scene.

This view of an object will be determined to a first approximation by the Form of the object including its size and position as well as its distance and orientation with respect to the conscious being. In creating a subjective view of an object, the brain is guided by the physical parameters which relate to that object and which are 'measured' relative to the brain/body. So while there is a good deal of creation going on by the brain in terms of the perception of and participation in a 3D world, there is usually (without the assistance of experience-changing drugs!) little scope for creativity: Objects as perceived will in general and to a large extent be the objects as they really are (a physical cube will be perceived to be a cube!).

So the objects we see and interact with are physical objects. It is our perspective of those objects, our individual viewpoint, which are creations of the brain. Our individual viewpoint includes a view of the body as an integral part of the visual scene, with the body as an object in its own right (whether or not we are conscious/aware of this). The set of all objects perceived is "the body plus all other objects in the visual scene".

Poster Paper - The Science of Consciousness Conference, April 2022

Core consciousness

The focus so far has largely been on consciousness in the context of visual perception. But obviously visual perception is not required for conscious experience; at a very basic level there is a sense of being or sense of self, a sense of embodiment. A sense of being a body (person) who just is.

A well known thought experiment by Daniel Dennett, entitled Where am I? raises questions about where the "I" actually exists: One's brain, one's body, or neither!

Embodiment seems to play a big role though combined with 'a sense of being'.

Poster Paper - The Science of Consciousness Conference, May 2023

A summary of the above ideas together with some new material was presented in a poster session at The Science of Consciousness (TSC) Conference 2023

Form as an extra ingredient

About the author

Paul's career started in physics and engineering research and academia. After a PhD investigating a possible cause for the magnetism of the moon's surface his first job was research with British Antarctic Survey into Very Low Frequency (VLF) radio emissions recorded in Antarctica. This was followed by 3 years as a lecturer in Electronic Engineering at the University of Sussex UK where his research involved applying digital signal processing techniques to detect faults in industrial sensors.

He left the academic world to start an independent IT business, run with his wife Behjat, and that lasted successfully for 10 years or so. Paul then spent a few years in IT management before transitioning to project and programme management and has earned his living in that profession since then. He is a chartered engineer (CEng) and a member of the British Computer Society (MBCS). Paul and Behjat have a grown up family now (with three grandchildren) and live in a rural setting in the Blackdown Hills, Somerset UK, where Behjat runs an exclusive holiday let business

Paul's interest in consciousness started around 23 years ago, firstly from a Strong Artificial Intelligence (Strong AI) point of view, that is, whether a machine could be devised which under the right conditions could actually be conscious. This would be a machine which has computational capabilities but would likely have a completely different underlying construction or 'architecture' from the computers we know. But he quickly moved on to human consciousness after taking the view that, given human consciousness already exists, a more promising line of investigation might be to start there!

Contact

If you have any comments on the site, or the subject of the hard problem of consciousness in general, please feel free to mail paulhurren@thistlecube.com. Feedback is very welcome.