From doodles to Descartes: sketching and the human cognitive toolkit | Judith Fan

Before the written word — and possibly even before speech — humans have communicated through drawing. From crude scratches in the dirt or on cave walls to the arcane symbology of the laboratory whiteboard, our instinct for conveying our thoughts visually is pretty extraordinary. 

We see or understand something in the world, we build an idea in our mind of what we think we see, and then using our hand and the utensil we re-create it to communicate the share our perception with others. Along the way, we add in our own understanding and experience to craft that communication in ways that might not correspond with a specific object in the world at all.

How we do this — and how we can learn to be better visual communicators — is at the heart of our conversation with Judy Fan, who runs the Cognitive Tools Lab in Stanford University's Department of Psychology.

We've been nominated for a 2025 Signal Award for Best Science & Education Podcast! Vote for us in the "Listener's Choice" category by October 9.

Learn More:

• Cognitive Tools Lab, Stanford Department of Psychology
• Fan, J., et al. (2023) "Drawing as a versatile cognitive tool." Nature Reviews Psychology. (pdf)
• Hawkins, R., Sano, M., Goodman, N., and Fan, J. (2023). Visual resemblance and interaction history jointly constrain pictorial meaning. Nature Communications. [pdf]
• Fan, J., et al. (2020). Relating visual production and recognition of objects in human visual cortex. Journal of Neuroscience. [pdf]
• Fan, J., Yamins, D., and Turk-Browne, N. (2018). Common object representations for visual production and recognition. Cognitive Science. [pdf]
• More recent papers

We want to hear from your neurons! Email us at at neuronspodcast@stanford.edu

Thanks for listening! If you're enjoying our show, please take a moment to give us a review on your podcast app of choice and share this episode with your friends. That's how we grow as a show and bring the stories of the frontiers of neuroscience to a wider audience.

Learn more about the Wu Tsai Neurosciences Institute at Stanford and follow us on Twitter, Facebook, and LinkedIn.

Transcript

Nicholas Weiler (00:00):

Hey, everyone, Nicholas Weiler here. Before we begin, we've got some exciting news to share. From Our Neurons to Yours has been named a finalist for a Signal Award in the science and education category. This is a big honor for us. And there's also a People's Choice Award where you, our listeners, can cast a vote. If you've enjoyed our show and want to support what we're doing, please take a moment to click the link in the show notes and cast a vote for us. It only takes a minute and it really means a lot. That's it. Thanks again and let's get to it.

(00:39):

This is From Our Neurons to Yours from the Wu Tsai Neurosciences Institute at Stanford University bringing you to the frontiers of brain science. Some of the greatest leaps in human knowledge started as simple drawings. Think of the first maps traced in the sand, the x and y axes that made abstract mathematics visible or the periodic table that laid out the building blocks of chemistry. Each of these began as simple marks on a surface, symbols that let us articulate relationships that had previously only existed in our heads. The impulse to draw, to sketch, to doodle is remarkable and distinctly human. When we draw, we're turning our understanding of the world, what scientists call our semantic knowledge, into a visual record that others can grasp and build upon. We sketch in order to explain, to plan, and to discover. Sketching is a form of learning and communicating and it's far more complex than it looks.

(01:50):

Enter Stanford neuroscientist Judy Fan. Her lab, the cognitive tools lab in the Department of Psychology, is asking what happens in our brains when we sketch and how can we better understand the interplay between what our eyes observe in the world, how our minds process that information and what happens when we try to put concepts onto a canvas.

Judy Fan (02:17):

So, we're interested in visual communication, the ability that people have to take some idea, take what they know, semantic knowledge about the visual world and turn it into some kind of physical representation that is meaningful. And we're interested in that phenomenon because it plays such an important role in human culture today and also requires understanding how multiple different mental processes are coordinated in the mind and brain. So, perception, memory, planning, action selection and motor control all at once.

Nicholas Weiler (02:59):

Yeah, it's funny because you think about like, "Oh, I'm just going to draw a picture of a chair," but actually you're like, "Okay, what is this other person going to recognize? How do I draw a chair that represents all these different chairs?" There's motor coordination like, "How do I actually make the lines?" and so, yeah, there's a lot going on there.

Judy Fan (03:14):

And that complexity is part of the point. Rather than trying to isolate visual processing, we are instead beginning from the premise that here's this really important behavior in human cognition and in daily life and we want to back out what are all the relevant processes that we would need to understand and how are those related to each other in order to give rise to that natural complex behavior. So, a lot of work in vision, which is where I got my start, really focuses on how the brain processes visual inputs. And when we study sketching, as you just said, we are interested in flipping that around and instead thinking about how people create visual inputs that are useful for some purpose, for example, to communicate some idea.

Nicholas Weiler (04:05):

So, I'd love to talk about a particular study that you've done. I'm thinking of your nature communication study from 2023. If I understand correctly, they're playing this Pictionary style game and people generally get faster and faster and, interestingly, their drawings got simpler but clearer. What are you trying to learn in that study about what people are doing cognitively when they're trying to communicate through sketch?

Judy Fan (04:31):

So, in that study, we had people get paired up online, complete strangers who had no other way of interacting with each other except through this digital drawing canvas. One person was assigned the role of a sketcher and the other person, their partner, was the viewer and the sketcher was shown four different images. In the 2023 study, we used this data set of office furniture, we used chairs that were rendered based on this 3D object database and we cued the sketcher to produce a sketch of one of these chairs so that their partner would know immediately which chair they had drawn. And we made sure that these were all chairs so that we could get more precision when it came to what information do people prioritize in the image to communicate given that all the other objects, all probably have some kind of seat back, some kind of support structure like legs or something like that.

(05:30):

So, we were really interested in first understanding how people decide what parts of the chair to emphasize in order to communicate the identity of the object they were drawing. And then, secondly, because we had these two people paired up over an extended period of time, we wanted to see how their strategies evolved based on previous interaction history. Meaning, if that sketcher was asked to draw that chair again, do they use exactly the same strategy the second time around, the third time around, the fifth time around or do they find some way of homing in on those details. If a particular chair has a funny armrest, zeroing in on that particular property of that chair in order to convey to their partner the idea that they had in mind. And so, that was the setup. And what we did was we collected every single stroke that the sketcher produced and when they drew it, how long they paused before they began to draw, when they stopped.

(06:29):

There was a version where the viewer could interrupt and buzz in, so to speak, and immediately guess and we also had a version where the viewer just had to wait until the sketcher thought that the drawing was done. We were really interested in seeing how the decisions that people make, especially towards the end of the session, might depend, not only on the immediately visible features of the object, but also, critically, memory for what had worked well in the past. So, if I tried something out, I made this drawing, I went for the armrest and that didn't work, what we found is that people compensated for that. On the other hand, if I went straight for the armrests and you got it, we found that people simplified and persisted with that strategy that worked. And so, really, that was a study that had the goal of understanding those dynamics.

Nicholas Weiler (07:24):

Right. It's a social learning thing, people are developing this shared visual language even though they've never actually met and talked to each other but they figure out, if I indicate this particular thing visually, you're going to get what I'm talking about. This gets to one of the things that I think is so interesting and there's this strange connection in your research between drawing, sketching and language, in communication. I don't think we usually think of drawing and art, let's say, as communication, we think of art as some sort of self-expression or something like that. But when you think about cave paintings, you think about the evolution of written language which all comes from symbols for things in the real world that gradually evolved and got simplified and simplified just to be shorthand. So, is this the thing that's driving your research, is this one of the things that's motivating this exploration?

Judy Fan (08:21):

Absolutely. And for what it's worth, language can be used to communicate, it's also used for expression, of course, it's similar when it comes to visual production. When people are creating visual representations, it can serve multiple different functions. I think a really important function is to mediate social interaction. It's really about taking some internal experience idea and putting it out in the world to share. And I think that what's really important about it is that, unlike other kinds of nonverbal communication modalities, I'm thinking of gesture in particular, these are really powerful activities and behaviors for conveying ideas and coordinating behavior. I think there's a lot of evidence that suggests that this is a fairly ancient behavior, how early modern humans coordinated their actions.

Nicholas Weiler (09:14):

Are we drawing maps on the ground with a stick? There's some way of communicating what we're going to do, we're going to go on a hunt or we're going to go find this place.

Judy Fan (09:23):

Exactly. It's when that idea can persist over time in the form of a durable physical representation, even if it's just dragging a stick across the sand, that's something that I think unlocks so many more possibilities for what people can do with each other and also what thoughts we can think.

Nicholas Weiler (09:40):

There's this great passage you have on your website where you're describing your work that really cracked this open for me. When you were saying think back to the 17th century when the coordinate system, X axis, Y axis, that was a big discovery, a big visual invention, a visual convention that lets you think about geometry and mathematics in a different way. And we have the periodic table of elements which lays out chemistry in a way that is almost instantly understandable or at least makes it much more possible to understand the whole system. It's almost impossible to imagine now understanding chemistry without the periodic table because you can't really think about it effectively without that visual. So, I guess my question is how do you draw the connection between these big idea diagrams, pieces of the cognitive toolkit, as you call it, and something a little bit more simple like these Pictionary games, drawing a chair and so on?

Judy Fan (10:37):

Right. So, first I want to mention that the reason why I think those examples from the history of science and math provide so much inspiration for the work that we do in the lab is because there's this puzzle that they pose. On the one hand, it took a really long time for us to realize you could organize knowledge about different elements into the periodic table. It took humans a long time to invent the Cartesian coordinate system. But once it was invented, it could catch on much more quickly and now we expect kids to pick up on this.

Nicholas Weiler (11:12):

Right. You teach it in elementary school, yeah.

Judy Fan (11:14):

And so, it's that duality. How can it be so slow and so hard to discover those kinds of formats for re-representing our knowledge and distilling it and compressing it into this really useful often graphical format and also expect that it can be taught in school and that everyone can learn how to make sense of and use these kinds of representations.

Nicholas Weiler (11:35):

Right. It's like these diagrams are unlocking a certain understanding of the world in a fundamental way and, once they exist, it's unlocked.

Judy Fan (11:42):

Yup. And with those examples from history of science, the challenge they posed to neuroscientists is how is it possible. Looking at the marks on a page that we know to be the periodic table or to represent the Cartesian coordinate system, how a visual scene that, okay, takes diagrammatic form is that once possessing properties that are somehow sculpted, designed in order to be legible immediately to the primate visual cortex. It's already set up so that we humans can make sense of it and break into it and learn about it quickly and that it's not enough to simply be a human and instead it requires practice and instruction in order to learn to use these tools.

(12:30):

So, the fact that these objects require really sophisticated machinery in the brain in the form of the primate visual system and everything that evolutionary processes over millions of years have endowed and the way that the brain is organized and develops to support really sophisticated parallel visual processing and, at the same time, also requires learning and memory and plasticity and adaptation and a particular kind of curriculum of experience. That kind of intersection and how those two different constraints interact with each other, it's a version of the kind of duality that's presented.

(13:10):

Anytime someone wonders about is that due to primarily nature or nurture, is it innate or is it learned, we study drawing and these graphical representations because the answer is obviously both and we get to focus on how it is both, why it is both, how the fact that it is both, what that means about how the brain is organized and how it works and what it means and what it looks like to have someone who's really fluent in these kinds of symbolic systems, what's different about that brain as opposed to a brain that's uninitiated.

Nicholas Weiler (13:44):

Yeah, that's why the phrase you use part of the human cognitive toolkit is so apt because humans are, let's say, evolutionarily or genetically tool users but you do have to use any individual tool. And so, to become great with a hammer or to become great with a computer or to become great with visual graphics takes a lot of training even though they rely on this underlying human ability to interpret abstract images and so on.

Judy Fan (14:11):

Yeah.

Nicholas Weiler (14:11):

I want to get into the neuroscience of this. It seems like one of the interesting observations that you've been making in some of the neuroimaging side of your work is that we have similar brain systems involved when we are being the artist and when we are being the person interpreting that art. And that's interesting because you wouldn't think looking at an image would involve the same brain systems as making the image. Can you tell us a little bit about what you've found there and how you interpret that?

Judy Fan (14:38):

Yeah, absolutely. So, the setup for the neuroimaging study was basically a Pictionary setup except now you have a participant lying down in the scanner and on their stomach is this plastic tablet where they can draw. On every trial, we cue them with an image, we used furniture again in the study so they might be shown a table or a bed frame or something like that and they always had about 30 seconds to create a sketch of that object so that somebody else would know what category it belonged to whether it was a bed frame, a bench or a chair.

(15:19):

What we did throughout every drawing trial is monitor the patterns of neural activity in early visual regions but also in multiple other regions, in parietal cortex, along the dorsal stream, it's sometimes referred to as, to understand not only the content of the neural representations in these sensory regions but also how those sensory regions were coupled with, how they were communicating with downstream regions that are thought to be really important for guiding action.

Nicholas Weiler (15:54):

Well, actually, on that point, let's take a little step back here and talk about the visual brain and how it's connected to other parts of the brain. You mentioned the dorsal stream. There's this idea that sensory information comes in through our eyes, goes into the visual cortex and then gets processed along two different streams. And you've got a ventral stream that comes down the lower side of your brain where your brain is interpreting what am I looking at, identifying these shapes add up to make a particular image. And then you've got the dorsal stream that comes up along the top of the brain and is more connected to how is something moving through space, how might I interact with this, do I need to catch this, what am I doing with relationship to this object.

Judy Fan (16:36):

Exactly, exactly. That's one of the most influential ideas in cognitive neuroscience for thinking about how visual inputs are processed and reformatted in order to support and guide behavior. Basically, our ability to make sense of the visual world around us does not happen instantaneously but is something that takes multiple stages of quite intensive computational heavy lifting on the part of the multiple regions in the brain that are hooked up in series with also a variety of other kinds of recurrent and what are called feedback connections as well in order to arrive at an internal representation of the visual input that is useful for behavior.

Nicholas Weiler (17:24):

I just want to hop in here because some of this can get maybe a little confusing. What we're talking about is that, when you see a chair and a researcher is asking you to draw it, you're activating a lot of different systems in the brain. There's perception, what you're actually seeing in front of you and that you're trying to recreate and that has multiple parts. What is it? Where is it? What's the shape like? How do you interact with it with your body? Those are handled by separate visual streams. There's also memory, we've seen many chairs before and that's knowledge you're bringing to the experience. And there are your motor skills, there's how you're able to translate what you're seeing in your mind's eye to the actions of your hand drawing that chair on a piece of paper.

(18:08):

When Judy is doing these experiments, one of the things she's trying to do is to see which of these systems are being called upon as you're trying to recreate this image that you've seen on a piece of paper to communicate with another person. Some of it is perception, some of it's a more abstract concept of the thing you're trying to draw. With new techniques though, we have new ways of observing all of this going on in the brain in real time.

Judy Fan (18:33):

Our ability to monitor patterns of neural activity in all of these different regions at once allowed us to ask what is happening in people's brains when they are drawing a table, okay? It's not a table that's in front of them keep in mind.

Nicholas Weiler (18:52):

Yeah. The concept of a table.

Judy Fan (18:54):

The concept of a table.

Nicholas Weiler (18:54):

Right.

Judy Fan (18:54):

We did show them a picture of a table to remind them what tables look like but then that image went away and then they had to draw from memory. And what we found is that the same pattern of neural activity in early visual regions that is evoked when people are looking at the actual table are reactivity, reinstated when people are drawing that table and that's true throughout the entire drawing process. And something that was also really intriguing is that evidence for that table which we collected using some machine learning tools to quantify the degree to which the power of neural activity in those early visual regions was similar to the activity when they were looking at the table.

(19:40):

So, that's how we quantify how much evidence there is for that particular concept, that evidence for that increased throughout the trial might reflect the simultaneous engagement of both an active, well-maintained visual working memory representations of the table that we showed them previously as well as the drawing that they're making on the canvas that is updating in real time as they make their sketch. So, we're seeing this convergence of both tableness that maintained in this working memory as well as the reinforcement of that signal from the output of motor actions, their drawing.

Nicholas Weiler (20:16):

Right. So, as people practice that physical activity and the visual recognition are coming together.

Judy Fan (20:22):

Yeah. And really driving it up the signal that this is a table.

Nicholas Weiler (20:29):

To make this a little clearer, what Judy's doing in this experiment is comparing a participant's brain activity to how it looked when they were looking at a picture of the object they're supposed to sketch. What she's found is that, as the person is sketching, their brain activity is actually getting closer to how it looked when they were actually looking at the picture. What that suggests is that the process of drawing is somehow clarifying that image in the mind's eye. The memory of the chair and the drawing of the chair are reinforcing each other to build a clearer picture.

(21:14):

So, it seemed like one of the interesting takeaways, particularly combined with an earlier study that you did, it seemed like one of the things you were seeing was that making sketches is actually making people better at interpreting sketches. Because I know that you had a study with Dan Yeamans who's been on the show talking about AI and mental models and things where you could see that, as people did that physical activity of making the sketches, they actually got better at doing some of that visual abstraction and seeing what someone else might be trying to convey through their visual language.

Judy Fan (21:46):

Right. So, that was an earlier study, it was with Dan Yeamans as well as Nick Turk-Browne at Yale Superfund. We were exploring this fascinating question that's really puzzled psychologists, neuroscientists, artists for a long time which is do artists see the world differently. Does all that experience painting and drawing and sketching change the way you see the world around you? Now, I don't think our study actually addresses that question, okay? But it's a fascinating question about the relationship between seeing and creating. And so, we took that into the lab and had ordinary people, people who don't identify as visual artists and we had them draw, again, familiar objects they knew about like dogs, trumpets, lions, cats, rabbits, these kinds of objects.

(22:35):

And then we had them practice making those drawings, these are full grown-ups, okay, so it might have been a few years since many of these participants had actually drawn a rabbit and then we wanted to know how that experience impacts their ability to make sense of ambiguous images. Okay? So, in the actual study, we used office furniture, okay? So, we make a lot of use ... For various reasons, it's a nice stimulus class that has a lot of nice properties. There's a lot of experimental control that you can have when you use manufactured objects as the stimuli but we generated these ambiguous hybrid furniture items, okay, so-

Nicholas Weiler (23:23):

You couldn't quite tell if it's a table or a chair?

Judy Fan (23:24):

Yeah, exactly, exactly. So, there were some images that were clearly ... This is a table that you might put on your patio, it's just a wooden table, the most generic wooden table you can think of, okay? And there's also benches, we had bed frames and then also a chair, just the most canonical versions of these everyday furniture objects and we worked with this artist, actually, to generate these morphs between each of these pairs of objects.

(23:58):

So, like you said, we synthesize an image that was exactly halfway between a bed and a bench or a chair and a table, not quite a chair, not quite a table, that was by design. And we showed people these ambiguous morphs and forced them to tell us what they thought it was. We know it may not be obvious but tell us is this a chair or a table? And we showed people these morphs that essentially interpolated between these canonical versions of each object.

Nicholas Weiler (24:25):

Right. So, you've got 100% chair, 80% chair, 60%-

Judy Fan (24:29):

Exactly.

Nicholas Weiler (24:29):

All the way to 100% table.

Judy Fan (24:31):

Yup. And we kept track of how often people assigned the chair label to this 80% chair, 60% chair, 40% chair and then analyzed how their categorization, judgements changed before the drawing phase and after the drawing phase.

Nicholas Weiler (24:51):

So, they do the categorization and then they do practice drawing these things and then they do the categorization again.

Judy Fan (24:58):

Exactly. And the key point of using these ambiguous morphs and sweeping through the intermediate ambiguous morphs between the canonical versions of these objects is that it allows you to measure how consistently, how ... This is a phenomenon that's known as categorical perception where continuous physical variation in some sensory input might be treated as categorically distinct.

Nicholas Weiler (25:24):

So, just to be clear here, the idea with this study was to deliberately bend people's brains. If you're looking at an object that's 60% bed and 40% chair and you have to make a decision about which one it is, you're naturally going to feel a little confused. That's the point because one of the things Judy was trying to do was to see if drawing the object actually helped to be a little bit more accurate in categorizing these deliberately confusing pictures.

Judy Fan (25:53):

So, before drawing, on average, as the amount of chair in the object increases, the likelihood that people assign the label chair does go up but the rate at which they do that, the slope, if you can imagine drawing this curve, as you go from zero to 20 to 40 to 60 to 80 to 100, that steepness of that curve increased after the drawing phase. In other words, it became closer to this categorical perceptual phenomenon. Now, I think that that study is one that is really, really intriguing and raises further questions about what explains that affect which we measure using people's perceptual judgments, their decisions, their categorization judgements. We can't tell the people we're literally seeing or experiencing that ambiguous morph had changed radically, profoundly after 20 minutes of drawing experience. But what we did find is that, the decisions that people made, they're more consistently taking ... If 50% is the cutoff, they're more consistently rejecting 40% chairs as not chairs and 60% chairs as chairs.

Nicholas Weiler (27:08):

Yeah. So, it's the idea we've talked about before on the show with reading about perceptual expertise. People are getting a little bit more just cued in on the key features that they're looking at which makes it easier to just say like, "Okay, I'm seeing legs, I'm seeing four legs, I'm going to call it this." So, just reflecting back on the studies we've talked about, we've talked about people getting better at creating these simplified, clear communication of visual identity as they practice sketching in your Pictionary game. The sketches get a little bit simpler, they get faster and they get better at communicating with each other. We've talked about the activation of the motor and visual brain coming together so that drawing something and interpreting a drawing seem to be activating some of the same circuits.

Judy Fan (27:57):

Can I share something else related to that finding?

Nicholas Weiler (27:59):

Yeah, please.

Judy Fan (28:00):

So, we were monitoring patterns of neural activity across multiple regions, early visual regions as well as more later stages of visual processing including those that are really thought to be supporting motor movement and action along the dorsal stream as we were talking about. We're trying to account for this behavioral effect that we identified in this earlier study with Dan and Nick. So, this categorical perception, that was a behavioral effect.

Nicholas Weiler (28:26):

Right. Why is it that people get better after drawing.

Judy Fan (28:29):

Right, exactly. So, then, the neuroimaging study, one of the main goals was to understand what is changing. As people are accumulating additional drawing practice like this experience, what is changing the brain that could be related to that behavioral effect? And so, something that we observed and a really intriguing exploratory analysis of the activity in early visual regions as well as these regions in parietal cortex, top of the head, is that the degree to which the patterns of connections between these visual regions and parietal regions were shifting over time, okay?

Nicholas Weiler (29:14):

So, now we're seeing the actual brain plasticity, what explains someone getting better at interpreting a sketch as they go through the process of making sketches themselves.

Judy Fan (29:23):

Yeah. So, we're after this question of what is shifting over the half hour or so that they're spending practicing drawing a table over and over again and asking how the strength. And this is a very subtle phenomenon but what we're asking is, not just how much activity there was in one region or another and not even necessarily how tightly coupled the overall activity in these occipital and parietal regions were, but actually the detailed fingerprint, if you will, of the pattern of how the different individual parts within each region were hooked up or coupled to a target region. And we were analyzing and monitoring that pattern of connection strength from the beginning to the end of the drawing phase. And something that was quite intriguing, for some pairs of regions, this connectivity pattern carries more and more evidence about the concept people are drawing towards the end of the drawing phase relative to the beginning.

(30:33):

So, in every region you can zoom in and ask how does its activity evolve over time. You can also look at the entire pattern of activation values within a particular region and track how all of those different activations are changing over time. You can ask how correlated is the activation value with the activation dynamics in another region in the brain. They might be actually far away, it's multiple synapses away, but you can ask about the strength of the correlation or statistical coupling between the activities in two different regions over time.

Nicholas Weiler (31:13):

So, we're saying how can we find a relationship between the activity in one spot in the brain and patterns of activity elsewhere in the brain so that you can start to make assumptions about could one be driving the other, for example?

Judy Fan (31:27):

Exactly. So, this approach has traditionally been used to measure coupling across two spots in the brain that are fairly large so full regions. And in our analysis, we took that same idea and applied it to a more granular, high resolution way of analyzing the activity in all the spots within a region, the strength and the pattern of correlations in a second region. It's a way of probing the content, the task-relevant information that we think is being transmitted between brain regions. For example, that they're trying to draw a table right now but, about a minute from now, they're going to be trying to draw a chair instead. And in order to develop a measure that's sensitive to the differences between how to brain regions might be talking to each other, so to speak, we might really need a measure that's more sensitive to the patterns, the information that's being communicated between each region rather than just the fact that these regions are active.

Nicholas Weiler (32:37):

So, what you're doing here is you're doing this very specialized brain imaging where you're able to look at different parts of the brain and draw relationships between patterns of activity in one place and in another in the brain such that you can actually get a sense of it looks like these particular areas which convey some piece of information about the leg-ness of something or the types of lines that we're looking at or whether some is a chair with a back or a table, there's some conceptual representations in the brain and you can start to see some of those patterns and how they influence other parts of the brain so you can start to get a sense of what kinds of information the brain is using in interpreting these visual images. So, using that approach, what's your takeaway from this study about how our processing is changing after we've done some practice actually getting in there and drawing these sketches?

Judy Fan (33:33):

Right. I think what we learned is there are at least two different ways it could have played out. One is that the internal representation of table-ness or chair-ness in early visual regions is itself changing when you draw. That drawing experience is directly intervening on how those visual regions represent table-ness.

Nicholas Weiler (33:55):

So, that suggests we almost see something differently after doing some drawing.

Judy Fan (33:59):

Yes, that would be more consistent with that idea. Alternatively, it's not that the way those early visual regions represent tables that's changing, it's how information about how to draw tables that is grounded in that stable representation of tables in early visual cortex is changing.

Nicholas Weiler (34:19):

Got it. So, either we're seeing things differently or we're just getting physically better at representing what we see.

Judy Fan (34:25):

At translating the contents of perceptual experience into a motor plan. And we needed this set of complex analyses in order to tease those different ideas apart.

Nicholas Weiler (34:37):

And what were you finding? Is it a little bit of both? Is it more one or more the other?

Judy Fan (34:41):

Yeah, yeah. So, what we found is that the representation of table-ness in early visual regions did not really change throughout the session. Now, these are one-session studies, this is not like going to art school. At the same time, what we did find is, from early on in the drawing phase towards the end of the drawing phase, that the strength of evidence for table-ness, the table-ness of the way that these visual regions were coupled to communicating with these downstream regions was increasing over time. There's more information that we as neuroscientists could pull out from the way these regions were communicating with each other, transmitting information, we suspect, between one another that was shifting over time which I think is quite provocative.

(35:30):

I think it's really this is raising more questions than I think that this observation, this particular analysis or result answers but I do think that it's consistent the idea that this drawing experience rather than altering visual representations as such is instead really refining and sculpting the way that these sensory processing regions are coupled to and talking to these downstream regions.

Nicholas Weiler (35:58):

To put this in plain language, the way I understand this is that it's not that people are seeing the table any differently as they practice sketching it but somehow they're better able to communicate what makes it a table to their hand so that they can draw it more effectively. It's like they're getting in tune with the table's essential nature.

(36:22):

We started by talking about the idea of drawing and sketching and you also study prototyping like building models and things as a fundamental part of our human cognitive toolkit and a lot of the research you're doing seems to suggest that even a little bit of practice does make people better at communicating visually. We're learning something about how that's changing the coupling between our sensory brains and our motor movement-oriented brains and some of the processes that are going on. Just for today, I know that there's more work that you're doing that I'd love to come back to and talk about another time but, for our listeners today, what is a takeaway for someone just about this idea of visual communication as a way of representing ideas that can really be improved by practice and can affect how we can communicate with one another?

Judy Fan (37:18):

Yeah, awesome. I think it's a total shame that so many of us get so nervous when we're expected to make a drawing in front of other people. I think that all of these different modalities, speech, gesture, body pose, facial expressions, whiteboard drawings as well to communicate and coordinate with other people and we should be making use of all of them and not be afraid to use all of them. There is a lot of evidence in the educational psychology and the learning sciences literatures too that suggest that, if you really want to learn something and really learn it really deeply, it helps to come at that idea, come at that concept through multiple different approaches.

(38:02):

If you are taking a physics class for the first time, there's a lot of new information, ideas and skills that you're learning all at once. You might think that just memorizing a formula or an equation is the only true kind of understanding, that definitely represents a kind of definitive way of expressing really important ideas in physics but so is drawing a diagram. All these different representation modalities play a role. They're important and constitutive of our understanding and knowledge of any subject we care about and also are part of the full repertoire of behaviors, modalities that we use and should make full use of when we're interacting with other people.

Nicholas Weiler (38:45):

Yeah. That's where, to me, we often think about art and drawing as something that's very specialized and leave that to the artist, you got to go to art school and I draw stick figures, they're not very good but that's ignoring an important part of our human evolutionary heritage that we can communicate with each other this way. Go to any science lab, you'll see science is done on whiteboards, that's where science communication happens.

Judy Fan (39:08):

Right.

Nicholas Weiler (39:09):

Well, Judy, thank you so much for coming and talking with us about your work. I'd love to have you back, this has been a real pleasure. And I'm going to do some sketching and see if I've got a Pictionary game-

Judy Fan (39:18):

I'm glad to hear it.

Nicholas Weiler (39:19):

... hanging around somewhere.

Judy Fan (39:21):

Yeah, yeah. This is a blast. Thank you, Nick.

Nicholas Weiler (39:24):

Thanks again so much to our guest Judy Fan. She's an assistant professor of psychology in the Stanford School of Humanities and Sciences. To read more about her work, check out the links in the show notes. And the next time you're playing Pictionary, just remember, the right doodle might just change the world.

(39:40):

If you're enjoying the show, please subscribe and share with your friends, that is what helps us grow as a show and bring more listeners to the frontiers of neuroscience. And one more reminder, please follow the link in the show notes to give us your vote for the Signal Listener's Choice Awards, it just takes a minute and it means a huge amount to us. Thanks so much. We'd also love to hear what you're thinking, what's working for you on this show, what's not, do you have any suggestions for topics you'd like to hear about from the frontiers of brain science. Send us a note in a comment on your favorite podcast platform or get in touch by email at neuronspodcast@stanford.edu.

(40:17):

From our Neurons to Yours is produced by Michael Osborne at 14th Street Studios. Sound Design for this episode by Mark Bell. Our theme music is by Christian Heiges. I'm Nicholas Weiler, until next time.