Do whales and humans speak the same language? | DW Documentary - VoiceTube: Learn English through videos!

Subtitles section Play video

Here, off Canada's Pacific coast, researchers are
hoping to make a long-held dream come true.
Behavioral biologists and IT experts have teamed up
to create programs aimed at deciphering
acoustic signals from animals.
Could artificial intelligence help to identify
patterns in the sounds made by marine mammals?
Will we soon start to understand
what it is that whales are talking about?
The coastal waters between Alaska in the U.S.
and Canada's Vancouver Island are where a group
of killer whales or “orcas” spend their summers.
A team of scientists from the DEEP AL research expedition
are preparing to embark. “DEEP AL” stands for
“Deep Learning Applied to Animal Linguistics.”
Computer scientist Elmar Nöth from Germany's
University of Erlangen has spent years working on
automatic recognition for human speech patterns.
Can the same methods be adapted to animal languages?
Underwater microphones embedded in
tubes serve as the expedition's ears.
For three summers, teams of computer scientists
and biologists have set out to record orca calls
and document whale behavior.
Rachael Cheng from the Leibniz Institute for
Zoo and Wildlife Research in Berlin is looking for
patterns between behavior and animal vocalizations
that could help to decipher whale language.
I assume they would exchange information.
They may have a very, very different system which
may not fit into our prototype of a language.
Around 300 orcas are identified as "Northern Residents"
— as they spend each summer along the coast
of Alaska and northern British Columbia.
They live in small family groups and are very communicative.
Do different families use different dialects?
And is it possible to discern the meaning of individual calls?
Here the researchers lower the
highly sensitive microphones into the water.
The eight hydrophones can record sounds
up to a frequency of 100 kilohertz.
That's far higher pitched than what is audible to the human ear.
Thanks to the network of hydrophones and acoustic triangulation,
the researchers will later be able to
calculate the positions of the whales.
To avoid disturbing the whales with engine noise,
the research trimaran is equipped with an electric motor.
While visibility is usually limited underwater,
sound waves are transmitted over considerable distances.
That's why a communication system
with loud calls is clearly beneficial.
They're very tight together, with frequent direction change.
Circling. And body twisting. Plus a lot of echolocation.
They are socializing, like you will see.
They bump into each other and rub, and frequent twisting,
and then jump onto each other.
It sounds like they talk about the plan
“What are we going to do next?”
Orcas only spend about five percent of their time at the surface,
which makes systematic observation quite difficult.
The research team uses drones to
document the behavior of the animals.
Diving expeditions with whales are prohibited in Canada.
The scientists are looking for the
smallest meaningful units of communication.
Which whale is calling, and which one answers?
Are some sounds repeated more than others?
Biologist Elysanne Durand examines the recordings.
Each call comprises a series of brief pulses which
sound like melodic curves to human ears.
Each shift in sound could be meaningful.
While recording the whale calls, the researchers also
document the behavior of the animals.
So there's four individuals in this group here that are circling,
there were four back there with the male,
there is one I can see out of the corner of my eye coming towards us,
and the mom and calf.
So that's 8, 10, 11 individuals minimum.
The more data available, the easier it is to "train"
deep-learning programs to decipher whale language.
It's therefore a major advantage for the researchers that over
20,000 hours of orca calls have already been collected —
more than for any other animal species.
Whale researcher Jared Towers has been tasked by the
Canadian government with observing different orca populations.
Back in the 1970s, scientists here began documenting
individual animals as well as their group structures.
A108 is right underneath the boat here,
here she is, she is gone off ahead.
Jared Towers has no problem telling the orcas apart.
This scratch here on A108 has persisted
for at least a couple of years now.
So the way that we identify the individuals no matter what
population they belong to is by appearance and
you get used to looking for certain features
on an individual killer whale.
The dorsal fins and the patches around
the fins have different shapes —
enabling scientists to catalog each of the Northern Residents.
Every family forms a lifelong bond.
What we are looking at with all these families
is an adult female leading the group,
and the fathers of their offspring don't play
much of a role in that family group.
Jared Towers works for a federal institution
that safeguards Canada's waters.
His former boss, John Ford,
was the first to distinguish between the disparate calls of the whales.
Ford's research revolutionized our understanding of the
communication system used by the resident orcas.
We are listening to calls of A-Clan whales.
That is the exciting part of underwater listening.
You are getting a window into their life that you would never see.
John Ford discovered that the whales use about 50 different calls.
Different families prefer different types of calls.
These were named “dialects”
and used to help identify individual families.
When they are making the various stereotyped signals,
it's simply to keep in touch with everybody in the group —
they exchange them, they are constantly
monitoring each other's location.
They know where they are because of their very directional hearing,
and they can monitor the behavioral state, the excitement level,
the arousal state of all the other animals in the kin group.
Northern Resident orcas visit pebble beaches daily.
You can hear them rubbing right now.
They're making socializing sounds and you can hear
the pebbles getting pushed around.
And they just rub all sides of their body.
Just in the shallow part of the beach.
Only a few orca groups worldwide engage in this sort of body rub.
This behavior is not genetic, rather it's a tradition passed on
within families of the Northern Residents, just like their language.
Back on the German-Canadian research boat,
it's a challenge to locate each family among the 300 individuals
that make up the Northern Resident population.
The orcas are constantly on the move in an
expanse of water the size of Belgium.
The expedition covers an area from
Vancouver Island to the southern tip of Alaska.
Seagulls indicate where schools of salmon might be.
And this is where orcas often hunt too.
Calls from the “A-Clan” can be heard on the underwater microphones.
The team tries to determine the position of the whales.
They compare the calls with the catalog compiled by John Ford,
but they encounter discrepancies.
The calls of class "N9" are used by different whale families.
Calls from the same class should be almost identical,
but these differ in length, melody and harmonics.
Human analysis so far has amounted to only a rough classification.
Looking at the spectrogram I am very sure that we can
achieve something that rivals the human performance.
Two whale families approach. What calls are they exchanging?
Here is that call. OK, let's just record this.
After I hear the call the back group surfaced.
Then I spotted the front group turning around.
The researchers are interested in which group is calling,
which one answers, and which sounds they're using.
Look here, I4 is approaching the boat. You can hear lots of calls.
That's N23 from the GI clan.
There is a lot of variation also —
and interestingly, here is the A23 family.
The calls look very different.
That's how people differentiate different matrilines.
If we have a lot of calls, we can try to train a classifier.
The programmers use algorithms or “classifiers”
to automatically analyze millions of whale calls in order to
compare recurring sound patterns with recurring behavioral patterns.
This same method is used to decipher the meaning of
individual words in foreign languages.
For Elmar Nöth and his team, it's no easy feat to automatically filter
out the weak orca calls from the constantly
fluctuating noise of the ocean.
By matching thousands of orca calls on the input side with a
target value on the output side, they train “neural networks.”
These layers of neural networks then learn bit by bit
which sounds are typical for orcas.
The purple peaks indicate a high
probability of having found a whale call.
It's the first time a machine has been trained to
automatically detect orca sounds.
Still, clustering calls into similar
sound groups remains a big challenge.
When comparing tens of thousands of recordings,
all whale calls are sorted by what they have in common.
But the cluster is still too crude to form reliable classes.
Further programming is required.
The team suspects that the Northern Resident orcas have a
complex language system and may even process more
intricate sound patterns than humans.
But how much is even known about the acoustic capacity of their brains?
This is Telegraph Cove on Vancouver Island,
where Elmar Nöth and Rachael Cheng have an appointment.
Only a handful of researchers worldwide have
expertise on whale and dolphin brains.
Neuroscientist Lori Marino is one of them.
Is this an orca brain?
This is an orca skull, a cranium and in the front you find the melon,
where they do the echolocation.
And this part right here, this large part, is where the brain would sit.
Human and whale brains generally share
the same mammalian architecture.
Both cerebral cortices are wrinkled and complex.
But the orca brain is 5 times bigger,
among the largest of any animal.
And unlike the human brain, the orca also has a paralimbic lobe.
The part of the orca brain that mostly
fascinates me is this part right here.
We have this special extra lobe, called paralimbic lobe.
It connects feelings, emotions and thoughts.
And it's so complex in the orca brain, it has become its own lobe.
That means a lot more of their communication has to
do with their feelings and emotion?
Communicating feelings, communicating emotions,
is very important to an orca.
Whales have a far larger acoustic area than humans.
Lori Marino's team also discovered a second acoustic cortex.
And now we know that they have also a sound processing
area in the same area where we process language.
Based on what you said how they process sound,
can we say that we need a much finer window when we analyze?
It's not just fine resolution that we need.
We need a way to do the analysis at a much more
complex level than our brains can do it.
New brain scan methods have revealed that orcas have a
far more complex acoustic system than previously believed.
Could the same be true for other marine mammals
that use sound for their communication and orientation?
In contrast to other orca groups,
the Northern Residents at some point stopped hunting
seals and other marine mammals.
Today they feed almost exclusively on the largest salmon in
Canadian waters, the Chinook.
Stocks of the fish, however,
have been declining rapidly since the late 1990s.
The Northern Residents have to travel increasingly long
distances to find the salmon that still remains.
And despite stringent fishing quotas for the species,
their numbers continue to decrease.
Jared Towers from Fisheries and Oceans Canada monitors if
people are complying with the fishing regulations.
It seemed to be that if Chinook salmon
aren't quite so much abundant,
the killer whales aren't getting enough to eat and that
causes populations to go down a little bit.
The Northern Resident Orcas are
classified as a "threatened species.”
When Chinook abundance is low,
they don't simply go and try catching other fish.
They have not found an alterior prey resource.
It is not like they start and go kill harbor seals,
just when there is not enough salmon around.
The substantial human appetite for salmon has
decimated stocks to critically low levels.
The ruins of the salmon fishing industry serve as a reminder
of the once abundant fish stocks.
Wild Canadian salmon were in great demand around the world.
The survival of the Northern Resident orcas is tightly linked
to the fate of the Chinook salmon.
Lance Barrett-Lennard is a geneticist and
behavioral ecologist at the Vancouver Aquarium.
He visits the resident orcas every summer to document their
long-term health and measure how their body size and
shape compares to previous years.
To ensure the images are comparable from year to year,
the drone hovers exactly 30 meters above the whales.
The group's calf is almost a year old —
and is still suckling from its mother.
The social skills of orcas are illustrated
by how they deal with their prey.
You've got a species that is highly social, that shares all of its food.
So they're compulsive sharers, killer whales.
When they catch a fish, one will eat half and head it off to
somebody else and have a bite, and then another one takes a bite.
The adults sometimes leave living salmon to the younger whales.
In this way, the juveniles learn to hunt their prey in a playful way.
At the Vancouver Aquarium,
researchers compare the images from different years.
One indicator of the whales' state of health are
the white patches above their eyes.
If there are not enough fat reserves in the neck area,
the angle along the eye-spots will change.
Comparing the images side by side,
researchers observe a life-threatening weight loss.
In response to the decline of natural stocks,
the fishing industry installed aqua-farms in the regions the
remaining wild salmon migrate through.
Nets protect the farmed salmon from orcas and other predators.
The predominantly Norwegian operators introduced
farmed Atlantic salmon species —
and with them, new viral diseases not
previously found in Canada's Pacific Northwest.
These fish farms raise as much salmon
as possible in a confined space.
It's all about maximizing profit.
The by-product? Feces and germs that escape into open waters.
There is growing concern of viral
outbreaks among the wild Chinook salmon.
If the primary food source of the resident
orca population continues to decline,
the whales' own survival will also be endangered.
Whale researcher Alexandra Morton is studying the
new diseases affecting the salmon.
She visits the research trimaran to share the results of her analysis.
I brought some results on my laptop,
if you'd like to see them, the virus research —
where we found it and why are the salmon farms such a
big problem for the salmon and the whales.
Mostly because of where they are located.
So we do a molecular test that basically just looks for the
genetic sequence or part of the sequence of the virus.
Salmon take in viruses through their gills.
The pathogens then infect their bloodstream, causing organ failure.
When the salmon farms moved in, the orca stayed away.
That made Alexandra Morton suspicious.
She took samples near the farms,
which were then analyzed in special laboratories.
She examined the organs of both wild and farmed salmon.
95 percent of the farmed fish she dissected were sick.
A large proportion of the wild salmon also became
infected through contaminated water.
The pathogens escape from the fish farms with the
tidal currents and spread to the regions the young
wild salmon have to pass through as they migrate.
200 kilometers further north, the “A42” family
searches for prey along the shorelines.
The research expedition has reached the fjords of Fisher Channel,
once famous for its ample fishing grounds.
The whales hunt by day and by night.
They don't need light for orientation.
Biologist Florence Sullivan compares her recordings to
known call types and notes down behavior.
Do the orcas tend to use certain calls in
a social context, or more when hunting?
All recordings are later re-analyzed in Germany
with the help of "deep learning" programs.
In their search for prey, the Northern Resident orcas now
often cover 80 to 120 kilometers a day —
along hunting routes that have been passed on for generations.
To identify individual whales and examine changes in group structures,
programmer Manuel Schmitt and biologist James Field are working on
automatic "fin recognition software.”
It works similarly to facial recognition programs.
The process of identifying whales by their fins and dialect was used
years ago in the famous case of an
orphaned calf known as "A73” or “Springer.”
When her mother died, the then 2-year-old Springer was
separated from the rest of her family.
Eventually, 300 miles further south,
the lone calf was found off the coast of Seattle.
Scientists could tell from her calls that she was a “Northern Resident.”
Springer was brought back and reunited with her family.
So while researchers have been able to
identify whale dialects for 20 years now,
understanding the “language of whales”
represents a whole new challenge.
The call systems used by different whale families change
little from one generation to the next.
Springer now has two calves of her own,
"Spirit" and "Storm" — roughly eight and four years old.
Orcas typically spend their entire lives with their mothers.
For the first two years they are suckled
and over the following eight years,
they learn what they need to do to survive.
Another orca group has joined Springer's family.
Together they search for salmon close to the shore.
Their hunting strategies vary depending on the local environment.
This gives them a wealth of experience that,
like their communication system, is passed down through generations.
Their group behavior during the hunt seems to be coordinated.
The whales exchange a stream of calls.
The orcas are still able to find enough food by
spending more and more time hunting.
For a calf, it's one of many lessons in salmon hunting.
For the scientists, it's a rich source of data.
Human activity greatly impacts the whales' habitat.
This continuous loud sound...
and now the orca calls become the background.
The sound of passing ships doesn't frighten the whales...
but the noise does interfere with their communication —
and compromises their hunting ability.
The Canadian government is investigating the impact of
ship noise on marine mammals.
We listen for their vocalizations,
which can cover a range of 10 km or so.
Man-made noise in the world's oceans doubles every ten years.
One of the concerns we have about vessel noise is its
potential impact on the
echo-location abilities of the whales.
The whales' vision is very limited underwater,
because of murkiness in the water, especially at night.
Typically in these waters they can't see more than a whale's length,
perhaps 10 meters max.
In quiet conditions the whales can probably detect a
Chinook salmon at a range of perhaps 200 meters,
but that might be significantly reduced by masking, by boat noise.
Similar to bats, orcas scan their surroundings by emitting
clicking sounds and listening to their echoes.
The noise from ships masks those echoes.
To reduce its own engine noise,
the expedition boat runs on an ultra-quiet electric drive.
It sounds like a N4 call.
But it's masked in the noise.
Here we have the boat's engine noise.
And there is an orca call here.
I have another recording with our engine
You can hear the harmonics. It's much clearer.
Still we have this electrical noise at 10 kHz.
Let me have a look at the engine.
Although electric engines are far quieter,
there's nevertheless some interference.
Rachael Cheng inspects the static noise
and examines the control cables.
Is it better now?
Much better now.
The sound of electric motors is still rare in the ocean —
which is perhaps why the ship triggers the whales' interest.
The boat stops in order to maintain the
requisite distance of 100 meters.
The curious orcas are not quite as cautious —
and inspect the boat and its crew.
The scientists wonder whether the whales might
use distinct calls for different boats.
In any case, there is immense curiosity on both sides.
Orcas are extremely social creatures — and love to touch one another.
I think this is the A23 and A25 group. Floating sideways.
Putting its fin on the water surface —
where others just mingle around.
Are their jumps also part of their communication?
Or just a bit of fun?
Many of their behavioral patterns are
still only partially understood.
This is also due to the fact that orcas
rarely roam close to the surface,
where humans can observe them.
Some distance away, another group of orcas suddenly appears.
Marine biologist Elysanne Durand wants to get an overview.
Which direction should I go now?
If we can have the whole group on our starboard side ...
This is a completely different type of orca.
There are maybe 4 transients off the island.
Unlike the resident whales,
these transient orcas don't feed on salmon.
They hunt other marine mammals.
Do you hear anything from these guys, Rachael?
No, I don't hear any vocalization.
But it seems they're hunting there, foraging.
In another contrast to Northern Residents,
transient whales hunt almost silently.
This is because dolphins, seals and other whales —
their natural prey — have excellent hearing.
That means, they know exactly when they're in danger.
The transient orcas also inspect a rocky plateau for prey.
? a tactic that appears to be successful.
We hear them vocalize now.
The blood at the surface reveals a kill.
The transients begin to celebrate their catch —
and now exchange many calls back and forth.
Acoustically they are very different from the residents.
I'm curious whether the classifier can pick up the difference.
The homes of the roughly 500 transients
and 300 Northern Residents overlap.
But their calls differ so much, it's as
if they're speaking different languages.
Dolphins and other marine mammals can tell the difference.
While they immediately flee from transient whales,
they rush intentionally toward the residents
as soon as they hear their calls,
to hunt and play with them.
Below deck, Elmar Noeth and Christian Bergler work on
the automatic call classification.
Here, hear the orcas? It shows here the detection of those signals.
We always have 2 seconds, 2 seconds, 2 seconds.
You can clearly see the incoming sound.
If there is an orca, they will be detected.
What I like is that the confidence is so high.
The algorithm is quite sure that it did find.
The automatic call detection works?
But the challenge is still to match one groups of calls with another.
In human language, the equivalent of these sound sequences
might be simple statements.
The Northern Residents consist of 32 families.
They typically use different call systems —
but are still able to communicate with each other when they meet up.
Looks like the other group is joining them.
The calls between the groups go back and forth —
announcing their arrival in a kind of greeting ceremony.
Do you still think they are saying, I am here, if they are so close by?
No, it's not the same call.
The last time it was really one call the
individual was repeating all the time.
Here they are doing combinations.
Meetings between family groups used to be more frequent —
when salmon stocks were more abundant, and they had to hunt less.
It seems to be more melodic.
Their sounds or vocalizations were a lot longer.
I would say they are more complex.
They exchange another series of calls
before the meeting slowly breaks up.
These sounds might signify "goodbye.”
But the researchers will need to compare many other similar exchanges
before they can interpret it more decisively.
There are thick banks of fog between Vancouver Island and
the mainland as the end of summer approaches.
Tracking down the whales is more of a challenge.
The researchers' only option is to use
acoustic localization with the hydrophones.
Under calm sea conditions the sound of the orca calls
can range up to 10 kilometers.
Elmar Nöth checks the location forecasts.
At 12 o'clock, 200-300 meters?
Over there at 1 o'clock — 1, 2, 3, 4.
Localization is also important in order to interpret the calls.
It's like 2 or 3 here, 2 or 3 there —
and then it's always the same call we hear right now.
So it's probably, "I'm here.”
And the other one recognizes, "oh, Fred is over there.”
What I would do is to cluster exactly those calls where
I'm convinced that I heard the same.
And then just from the sequence one could make the assumption
that's a signal-and-response signal, something like that.
The last of the three expeditions is coming to an end —
with data from a total of 20 weeks of field research on board.
A few months later, the researchers meet at the
Pattern Recognition Lab at the University of Erlangen
in Nuremberg, Germany.
Rachael Cheng has an appointment with the programming team.
Manuel Schmitt has run an automatic comparison of one million calls.
Hi Rachael, good that you're here.
Take a look at the clustering. I made some changes.
It's a little bit better than before.
N3 is here, it looks very similar, but its mixed with N9s over here.
This one looks very different but it still needs training.
Background noises hinder the automatic sorting of various calls.
A newly programmed noise filter increases the precision.
Is that the new clustering? That's a lot better, right?
It makes tiny differences.
There are tiny artefacts.
There are still small errors. But on such a large scale —
where millions of calls are being automatically compared —
these become less significant.
I think we're now at the level where we can run
through a couple of tapes now.
But you can use those cluster sequences in order to find
those language patterns - like a semantic structure?
And those can be interpreted as a call,
followed by an answer-call, followed by another call.
The new clustering results are better than expected.
Rachael Cheng's task now is to assign
recurring call patterns to certain behaviors.
She compares the new clusters with the existing call-catalogs —
and discovers reappearing matches that might be meaningful.
The algorithms have learned to
differentiate the calls of different orca families.
That would normally take human researchers years of training.
In the future, this will allow automatic
detection of which family is calling,
and the ability to follow the
subsequent dialogues between families.
The next step is comparing millions of call sequences.
Which calls appear together frequently,
accompanied by which similar behavior?
When decoding human languages,
such connections might reveal meaningful possibilities such as
“sit” and “chair” or “table” and plate.”
The deep-learning programs cannot yet work completely independently.
Without human control, there could be
a huge increase in assignment errors.
The first matches appear
The same call sequences were used in a similar context.
But there are exceptions.
The comparisons made so far are not conclusive.
The re-occurring call sequences point
to the scene — they are socializing.
And we also find it the second time in a
socializing context, the same call patterns.
We expect to find it in the third sample
also, but we did not find it in this.
That does not disprove it. There could be different explanations.
We just don't have enough data to say that.
The German-Canadian research team is not yet able to create
a kind of dictionary of the orca language.
But they do now have tools to compare whale calls in a more
detailed and systematic way than was previously possible.
And as more recordings of whale calls are shared and
made available for training deep-learning machines,
the faster it will be to recognize the
subtleties in communication patterns.
Humans have long faced limitations when
it comes to understanding what animals might be talking about.
But with the help of AI, a new era of research might allow us to
decipher the secrets behind their communication.