Intelligence in video games. Since the beginning of this channel I've explored a variety of

AI techniques to solve specific problems within video game design that require both immediate,

reactive behaviour in conjunction with long-term and strategic decision making. First-person

shooters are certainly a common genre from which interesting AI can emerge given its

predicated on fast, reactive and engaging gameplay. But if we're to consider the need

for effective systems that balance both short and long term decision making in games, there

is no greater problem space to explore than Real-Time Strategy (RTS) games.

Both the RTS and FPS genres have continued to innovate the state of the art in AAA video

gaming in an effort to satisfy their design. But RTS games have much larger considerations

for the long-term impact of quick and reactive decisions, as well as the need to manage resources

at varying levels of abstraction and complexity over multi-hour gameplay sessions. When considering

the challenges and innovations faced in AI for strategy games, as a player, developer,

or researcher there's one franchise you simply cannot go without discussing, Creative Assembly's

Total War.

Total War has grown from strength to strength since the launch of the franchise in the year

2000. As each game becomes more ambitious, the challenges faced by the AI systems continue

to increase in scope and complexity. This has resulted in a development team that continues

to push the state of the art in game AI unlike any other in video game history. But the journey

to reach more recent entries at the time of this video, such as Total War: Attila and

Warhammer has had more than a few bumps in the road along the way. So in this first in

a series, I will be looking at not just how the core tenets of the AI of Total War, but

how each entry in the series aims to solidify, tweak or completely rewrite parts of these

systems. In addition, I'll look at the response to these changes from the fanbase, the modding

community that has thrived within Total War and how The Creative Assembly adopted groundbreaking

academic research to help innovate from the very first entry all the way to more recent

releases.

So let us begin our march to total war.

But first let's get into the Real-Time Strategy genre and understand where Total War sits

within this space. Real-Time Strategy games are focussed primarily on the control of territory

and assets: whereby two or more opposing forces will appear in the same map in an effort to

assert dominance by securing resources or locations within the existing defined area.

Often in order for this to be achieved, each force will build collections of structures

and units that either improve the resource collection within a given area, defend existing

structures and land or scout and secure areas of the map by force. The latter often requiring

the destruction of opposing forces units in the process. This process typically requires

players to explore a previously hidden environment by piercing what is referred to as the 'Fog

of War' in order to establish enemy and resource locations. In addition, over time each faction

is expected to build upon the structures and units previously established by enhancing

their existing capabilities and resource acquisition.

While arguably not the first RTS game ever made, Westwood's 1992 release Dune II: The

Building of a Dynasty acted as the archetype for future games in this genre such as Westwood's

own Command and Conquer series, but also Age of Empires, Total Annihilation, Dungeon Keeper,

Homeworld and Blizzard's Warcraft and Starcraft franchises. The genre has continued to evolve

and branch off since the late 1990's, with the likes of real-time tactics games like

Dawn of War and Star Wars: Empire at War emphasising tactics over resource management. Meanwhile

games such as 1994's UFO: Enemy Unknown have explored a balance between the real-time and

turn-based decision making, with it still proving popular to this day with the franchise

being successfully rebooted as XCOM in 2012. Arguably the largest deviation is the the

rise of the Multiplayer Online Battle Arena or MOBA genre - popularised by the Warcraft

III mod Defense of the Ancients and typified by games such as Dota 2 and League of Legends

- where players assume the role of hero units on the ground to define and assist the larger

strategies playing out during a match.

AI applications within real-time strategy games have existed since the inception of

the genre, with players responsible for managing the movement of faction units by telling them

where to go, but without direct control of their actual movement. As such, players are

expected to interact with these systems in order to yield desired outcomes. In addition,

AI is critical to the single-player aspects of the game, given that players play off against

opposing commanders responsible for managing their own units and construction strategies.

The role of the AI player is one that is tremendously challenging and is considered to be one of

the larger challenges facing academic research. This has led to the forming of numerous academic

benchmarks and projects, including the Starcraft AI Competition hosted at the AAAI conference

on Artificial Intelligence and Interactive Digital Entertainment. This is a video in

and of itself, but don't worry: I'll be talking about that on AI and Games in the near future.

So what makes Total War so special and challenging for an AI system? Creative Assembly's strategy

series is interesting in that it combines multiple modes of strategic gameplay into

a single game: with players managing turn-based resource management and political strategy

alongside real-time combat with large scale armies. In addition, the combat juggles between

micro and macro levels: requiring not only AI control of individual units, but more abstract

control of unit types and managing the combat formations and troop deployments on the battlefield

at runtime.

The first entry in the franchise, Shogun: Total War, balances the combat simulations

that strive for realism and authenticity, alongside the political strategy that is aimed

to give context and stakes to each conflict. The original Total War is set in 1530, during

the Sengoku Jidai period of feudal Japan: a time largely popularised in contemporary

fiction by the works of Japanese film director Akira Kurosawa, with films such as Kagemusha,

Seven Samurai and Ran proving influential on the design and development of the game

- with clips of the latter being used as part of the games cinematics. Both the player and

opposing AI assume the role of the Daimyo: local-lords who control provinces of Japan

with a need to conduct both diplomatic strategy alongside military movements. When rival factions

are drawn into conflict, players take control of the Taisho or general and move hundreds

if not thousands of troops across the battlefield. Sengoku Jidai made for an ideal period of

history for the game, given the politics and even the economics of the period was built

around the logistics of fielding armies in defence of the Daimyo and his ambitions.

Total War deviates from many traditional real-time strategy games in that it removes mechanics

such as Fog of War from combat gameplay as well as the need for resource management outside

of unit counts from combat decision making. Given the nature of the game itself, the AI

player is comprised of two distinct AI systems: the Diplomacy AI and the combat AI.

The Diplomacy AI manages the turn-based strategy of the game and is responsible for moving

armies around the map, conducting diplomacy (be it by sending envoys or assassins to forge

alliances or eliminate opposition) and building the agriculture and infrastructure of owned

provinces.

Meanwhile the combat AI dictates combat unit formations, strategies and attack patterns.

Much like human players, this is only responsible for managing the strategy of specific groups

of units, given the units themselves are already controlled by AI techniques.

So to get to grips with the AI of Total War, I'm going to explore the AI systems from lowest

to highest level: from individual troop control systems all the way strategic systems that

seek to conquer all of feudal Japan.

Total War combat is driven by units: groups of specific troop types that can be deployed

in formation ranging from melee types to archers to cavalry. These combat units are not only

expected to keep formation, but to move around and conduct combat as a unit. Movement is

often a tricky task, especially when asked to navigate through or around a variety of

terrains such as mountains and forests. To achieve this, Total War adopts artificial

neural networks for managing individual units. Neural nets are a fast and effective means

to achieve quick and reactive responses to a pre-defined objective. We typically train

them using machine learning methods but they can also be tweaked by hand if the network

is small enough. What makes this so effective is that once they are trained, the processing

time to make decisions is lightning fast. In addition, a well-trained neural net is

able to generalise its decision making process: meaning it can recognise the similarities

in numerous individual circumstances and in each case give a similar answer. Despite this,

neural networks are typically not great when being given multiple objectives to complete

at once, especially when they oppose one another. In this instance, the troops have different

neural nets they can trigger in depending on the objective they're trying to solve,

whether it's moving formation, avoiding oncoming enemy fire or taking position to attack themselves,

be it up close or from afar. Each of these networks is pre-defined and is not tweaked

or optimised during gameplay, so it's not learning to be better at the game as you're

playing it.

These Unit AI systems are in effect regardless of whether a human or AI is in control of

the army, as each requires these neural net systems to manage the individual troop behaviours.

The combat level is the AI system that controls opposing forces akin to players. This AI needs

to decide the group formations and actions of units. Moving segments of its army together

in order to achieve a specific tactic or micromanage movement of units given the terrain or weather

that is currently in play.

To achieve this, Shogun employs a logic system that allows for specific decisions to be made

based upon the current state of the world. These decisions are derived from 'The Art

of War': a collection of strategies by Chinese military strategist Sun Tzu that dates back

to around 500 BC. The use of a logic system is well suited to adapting the texts, given

that in many instances a set of conditions are in play that should result in a particular

response from the system. These rules considers the type, condition and size of both our units

and our opponents, the current terrain, the weather and state of troop morale. It's been

stated that there are 220 rules embedded in the combat AI system across three distinct

versions, each with their slightly different sets of behaviours.

A driving force behind this was to ensure that combat was as realistic as possible,

deviating away from the tactics that had driven many real time strategy games of the era.

As Mike Simpson, Development Director of Creative Assembly stated:

"Most [games] use simple scripts of canned behaviour that fire when you bump in to them,

and very simple swarming behaviours. They're limited, and are “gamey” rather than real

world. What I mean by that is that the tactics you use to beat them are something that you

have to learn for each game or sometimes each scenario/level, and bear no relation to reality.

What we strive for is a game where real world tactics actually work. It's not the easiest

path to take, but it is the most rewarding."

The final layer of Shogun's AI systems is the diplomacy system. This is in control of

all decision making on the main campaign map: managing army movement, conducting diplomancy

between factions and building infrastructure within provinces. This uses a mixture of classic

state-machine decision making alongside the use of genetic algorithms in order to customise

the tactics made by each Daimyo. Genetic algorithms are a form of machine learning where we create

large quantities of candidate solutions to a given problem that learn to solve it by

breeding the best ones with each other. These candidates aren't code, they're an encoding

of how the AI behaves typically as a string of numbers, which is then translated into

the AI itself to dictate how it behaves. These allows for us to create a collection of diverse

solutions for a given problem: they don't all behave in the exact same way and have

their own slight characteristics. As a result, each Daimyo has a unique combat behaviour.

From a players perspective, this gives them each some sort of personality: allowing for

Daimyo's that are more willing to conduct diplomacy to others that will take a more

aggressive stance in expanding their territory: attacking both AI and human player alike.

With these three systems, the core tenets of the AI of Total War are in place: to manage

individual units whilst retaining cohesive behaviour, to group units of troops together

in a manner that is responsive, reactive and challenging towards players and to create

strategic opponents with personality. Despite being built back in the year 2000, the core

principles of combat and strategy, combined with the systems built in Shogun have carried

the Total War franchise forward into new and varied theatres of war in many a subsequent

release. Despite these innovations behind the scenes - even from the very beginning

- the AI has never proven infallible and as such, with every release there is a drive

to tighten it up and improve it as best as possible.

The first changes came as the series migrated from feudal Japan in 2002, with the launch

of Medieval Total War. Medieval pulls the game farther back in time to the late 8th

century all the way to the 14th century, with players taking charge of a variety of factions

at different periods of history: with the likes of the Viking era in Britain - where

much of the country is fractured following its abandonment by the Roman Empire - all

the way to the late-medieval period, where much of Europe has taken shape as actual countries.

Medieval: Total War is in many respects a refinement of the original game and is built

on an improved version of the original Shogun engine that among other things allowed for

larger battles than its predecessor. There are no major innovations between Shogun and

Medieval for the underlying AI tech, just a number of optimisations and improvements.

The unit AI systems were improved to prevent bottlenecks in tight spaces such as bridges.