A more diplomatic AI

Making computer players better negotiators

from JoeUser Forums

One part of the "AI" isn't really AI.  A lot of people think of the trade screen in the game as being "AI".  But it's not. It's basically just a dialog full of weighted values.

The real AI may, for instance, call up the trade screen. But the actual valuing of items in the trade screen (money, techs, ships, planets, etc.) isn't in my area (the AI) but is instead just a table of values to weight everything.

Which means, of course, that it's exploitable. 

Creating a negotiation screen like this is non-trivial. 

For instance, in GalCiv, you can research diplomacy techs and get the super ability Diplomacy which gives you a huge advantage in negotiations.  But at the same time, we can't let people trade say 100BC for 101BC or else they can really exploit the game.  And that's the kind of thing that is more difficult to keep from happening than one might think.

In Civilization 4, the solution was to pretty much make it so that you could only trade apples for apples (techs for techs for instance).  That way, players couldn't easily exploit the AI.

In GalCiv, we slog through with trying to let people trade ships, techs, money, treaties, war proposals, etc. for any combination of those things. 

But the downside is that some people will find exploits.  There is no practical solution other than to remind people that it is a single player game and if they want to cheese the computer opponents, that's really their busines. 

We also try to do updates to the trade screen "AI" so that it is better in negotiating.  But there are just so many variables to take into account. 

If you have a particular "stupid" trade screen behavior, post below and I'll take a look at it. ;)

63,129 views 37 replies
Reply #1 from GalCiv2 Forums Top
Peace treaties. All the time you'll be slaughtering an AI player and the AI on the trade screen will be begging for peace; but when you offer Peace Treaty for Peace Treaty, he won't take it. At least, not until you throw in half your Trade Goods--and that's when you're WINNING THE WAR.
Reply #2 from GalCiv2 Forums Top

Peace treaties. All the time you'll be slaughtering an AI player and the AI on the trade screen will be begging for peace; but when you offer Peace Treaty for Peace Treaty, he won't take it. At least, not until you throw in half your Trade Goods--and that's when you're WINNING THE WAR.



I've run into the same thing more than once, and it's somewhat frustrating. They offer you something of theirs that you don't want, in addition to a peace treaty, you refuse, then suggest peace treaty, PERIOD, and they refuse, let alone peace treaty and something else.
Siege of Centauri
Reply #3 from GalCiv2 Forums Top
Peace treaties can go the other way too, though. It may be a hallucination on my part but it seems like you're more likely to get one the longer the conflict has stretched on, no matter what the relative performance of the sides. More than once I've managed to white peace my way out of certain doom, which never really made much sense to me. Heck, sometimes when they're casually slaughtering you they'll up and offer you a straight peace treaty. I don't really think of it as all that serious of a problem, but it's decidedly odd.
Reply #4 from GalCiv2 Forums Top
Is this "table of values" the same for all opponents in all situations (weighted by diplomacy) or does it change depending on the strategic goals of the AI's?
Reply #5 from GalCiv2 Forums Top
I agree with the previous posters, the AI is very poor at deciding the value of peace and war, when it should keep fighting and when it should want to stop. The Terran Alliance is especially bad about peace treaties, wanting too much for peace when it is losing, often leading to it's doom.

Also, I often have problems with apples to apples trading. No matter my diplomacy, the AI should want to trade me Phasor II if I offer it Phasor VII in exchange. I know this means its trading weapons tech... which it now recognizes as dangerous. It still will always gain from this trade, no matter the difficulty level or situation.
Reply #6 from JoeUser Forums Top
I have noticed that with a high enough diplo, the AI is willing to trade away it's last planet. This happens more often during a peace negotiation, but it ought not happen at all...
Reply #7 from GalCiv2 Forums Top
Also, not sure if this has to do with the real AI or the values table that you deal with, but the AI does not try hard enough to get multiple treaties. Every AI should want every treaty from all the other civs, those are free bonuses to your empire. The AI is lethargic at picking them up, and only ever seems to pick up two.

Part of this appears to be how they value the treaties. IMO, they are valued too low... I should be able to sell mine for more, and it should cost me more to get one. A treaty's value is a function of the time that it is up. Say I'm producing 200bc of research a turn. My treaty will now give 20bc of research per turn. So how many turns will it be active? Say, if you just want to guestimate/average a treaty will last for one year - 48 turns - then my research treaty is worth about 960bc. Sure, it might not last a year, but it also might last 10 years. Rather than trying to estimate the life of a treaty, I would pick a default "average" length the AI assumes a treaty will be up and then calculate the value accordingly. Often (especially in the first few years) I can't sell an economic treaty for 200bc, when my income is over 4k per turn. The AI would see an IMMEDIATE cash increase to take that trade, on the very next turn it would get 400bc back on its 200bc investment. Still, it stubbornly says... no deal. This is to the AI's detriment.

If it matters, I play on suicidal and on average on medium maps, so my experience and conclusions come from what I see on that level of gameplay. What I see might not necessarily hold true for other game environments.

- Edit - Also, generally income and research levels rise through the course of the game. Therefore, figuring the value of the treaty based on current values still probably undervalues them. Say, in my example above, I personally might value the research treaty at closer to 20*1.3*50 = 1300. This formula is just how I personally value treaties when I am trading in my own games, and is used really only for example purposes - not necessarily as a benchmark to shoot for. True, sometimes research and income values can fall, but on AVERAGE they increase, so that is what I consider, hence the x1.3 multiplier.

- Double Edit - For instance, this is a screenshot from a very recent game of mine on suicidal. I was playing as the Altarians. Why does the AI not try and do this??
Reply #8 from GalCiv2 Forums Top
Actually, I have more to say about trading in general on the top difficulty levels. The AIs approach seems to be to just value everything more highly, and this leads to it missing opportunities.

For instance, lets go back to that research treaty from a civ that is researching 200bc/turn. Say on suicidal the AI just values everything it has twice as much. Now, it wants 2600 for its research treaty. That is fine, it makes the game harder on me. BUT it will only pay 750bc for my treaty. But, the value of the treaty is actually 1300. Solution, I just don't sell my treaty. I don't have the cash, but the AI is the net loser, because my treaty really was worth at least 1300bc. Because the AI insists on paying too little for my resources, it consistently misses out on opportunities. This is especially true because I often become one of the most powerful civilizations, so what I can offer is usually quite a lot. Instead, I become more powerful by just keeping all of my toys to myself because the AI asks too much for me to be willing to share. This is exactly why the general feeling among players is that turning tech trading off makes the game easier. Not selling anything usually is an advantage. The AI should work harder to procure deals, exactly because it can make it more powerful.

Sometimes I might be willing to sell a planet that I get deep in enemy territory because it is too far away to defend. But if the AI won't pay for it, then it is just easier to take the diplomacy hit from close borders and alarming influence, and maybe invade the stubborn AI rather than selling it the planet. It should realize that offering to sell a planet is a huge asset. It should also realize how much of a loss selling a planet is to its overall potential. The AI being unwilling to pay for assets is only an advantage to the AI if I have money problems. If I am self sufficient economically, it is just missing out on opportunities.

This goes to exactly the problem with peace treaties. The AI is valuing its money over its survival, and so it gets conquered. True, if it thinks you will just attack again in the next turn the peace treaty is worthless. But what about when you honor your treaties? If the AI is losing a war, regardless of its military rating, it might be time to think about regrouping and suing for peace.

--
Another trading oddity. "Worthless preceding tech" is only worthless if the preceding tech doesn't include bonuses. Soldiering tech comes to mind. If I offer Planetary Bombardment with Advanced Troop Modules, the Planetary Bombardment is NOT worthless preceding tech. It confers a +5 soldiering bonus and gives you more invasion tactics.


Edit - And more along the same lines with the AI not working hard enough to trade more. Why doesn't the AI send me offers to buy planets that it finds offending? Why doesn't it approach ME about trading treaties early? Plenty of times I have bought ships off of the AI for various reasons. I have never once ever seen the AI approach me about buying one of my ships. If I offer something, it will think about it, but I'm the trade "aggressor", I'm setting the terms and making the deals. Why isn't the AI looking to trade with me for what it needs, why isn't it initiating? Especially the Terrans. You would think having them in the game they would early on be getting most of the treaties, and bugging me constantly about buying something new. Same with the Korx. But no, they just sit there, and the Terrans will just stubbornly demand more for any deals.

Don't get me wrong. I'm impressed by the AI generally, and I love this game. Trading and the AIs approach to trade has always seemed like one of the biggest holes, however.

Double Edit - imagine for instance, if the Terrans kept offering you good deals for your top tech. You don't want to make them more powerful, but its just such a good deal... There is so much more to a trading empire than just sending out a few freighters. You finally research large hulls and everyone else is using small or mediums. Suddenly the Korx approach YOU and offer you thousands of bc for your large hulls. Man that is a lot of money. How much can they do with a tech advantage really... Only, they can do plenty. This is the sort of approach a player takes, especially when building a trading empire. The AI on top levels makes unreal amounts of cash. To turn that cash into power, you want to play off of the players natural greed by offering them short term benefits (money) for long term problems (the AI now has your best tech). When the AI always just offers you bad deals, it is easy to just turn them down.
Reply #9 from GalCiv2 Forums Top
The AI also seems to value tech in a vacume. If everyone is using warp drives, New Propulsion Techniques really isn't worth much. I've had problems trading Extreme Colonization for Xeno Biology. What? What is the AI thinking? Why is it valuing those old techs so highly?

The value of any tech in this game is relative to the top tech reached in that tree at the current time in the current game. Having the top tech of any tree, except maybe sensorsIV and starbase fortification techs, is worth plenty. How much are research centers worth? If everyone is using basic labs and xeno labs, they are worth a lot. If everyone is using Discovery Spheres and Neutrality Learning Centers, Research Centers tech is practically worthless.
Reply #10 from GalCiv2 Forums Top
I'm currently playing my first game with tech-trading enabled in a long time.

The AI never sent me any acceptable trade. They kept insisting in wanting to trade my Alliance against their Missile weapon theory...

And this isn't limited to tech trading, since I never got offered a treaty or an alliance (in all the games I played), even though I see the AI trading it to another AI. So I'm pretty sure that the AI treat the human player differently than the others AI.

So my question is : Why does the AI never propose acceptable trade to the human player? I would be pretty happy to enable tech trading if the AI was to send me acceptable trades sometimes...
Reply #11 from GalCiv2 Forums Top
Treaties should be bilateral only, the above screenshot is why.
Reply #12 from GalCiv2 Forums Top
Wyndstar, your points are very well made, especially when you show how the AI is actually hurting itself by refusing reasonable trades and very offering anything reasonable.

After the patch that made the AI's much more stingy, I have the impression that in order to fix the "exploit" of trading to the AI's disadvantage, the AI's were made so greedy that they not only don't trade to their disadvantage, they also never trade to their advantage.
Unable to tell the difference, the AI will turn down a great deal for fear of being ripped off.

At what cost? Obviously, players are going to figure out ways to rip off the AI no matter what, as Draginol said. So is the game more fun with a stingy AI that drops you out of the suspension of disbelief because it turns down obviously advantageous trades or is it more fun with an AI that lets itself get ripped off a little more often but generally is pretty reasonable?
Reply #13 from GalCiv2 Forums Top
Treaties should be bilateral only, the above screenshot is why.


But, but, I like my "tributes". The thing is, it wouldn't be an exploitable advantage if the AI just tried to do it as well. I think the current "free" treaty system can work, where some empires get six sent there way, others get none. But for it to work, the AI has to actually want them.
Reply #15 from GalCiv2 Forums Top
I think that treaties should be re-negotiable. After a certain minimum contract length, the AI examines their worth and decides whether it's worth haggling for a better deal. A civ which is behind technologically should make a better prospective buyer for a research treaty.

By subtly changing who they grant a research treaty to, a civ can try to keep the playing field level instead of allowing one civ to suck off all the rest.
Reply #16 from GalCiv2 Forums Top
Draginol, your subtitle is 'Making computer players better negotiators', and you mention adding some real AI to the trading process, so I will stick to that for this post.

I have always wondered why the computer players don't consider the overall galactic situation when trading with the human player. I would like to see what would happen with trading if each computer player came to the table with an assessment of the current division of the galaxy, the human player's alignment and past actions, and personality quirks of their own.

I would call it AI situational awareness.

If you're trading with a neighbor who probably shares planets in the same systems as you, it would make sense for that player to completely refuse to trade their weapon and colonization techs. At the same time, it would make sense for them to be more willing to trade econ/research treaties to you, making it less likely that you would attack them and breach the treaties.

On the other end of the spectrum, consider a player who is on the other side of the galaxy. You have neighbors, i.e. potential enemies, in common, and you are not an immediate threat. They might be more willing to trade colonization techs and other things that could serve to weaken their neighbors.

I can't really tell if alignment is much of a factor currently. Diplomatic standings do seem to matter a little, but it would make sense for your alignment brother on the other end of the map to be more willing to trade with you, at least. Or, if you're the Altarians trying to trade with the Drengin, they should be a lot less willing to even buy your stuff.

How to decide how close is too close? Right now, human player influence on even a single computer player planet causes a lot of friction. The Thalan home systems are a hundred parsecs away, but that one radioactive planet they colonized in the middle of your empire makes them hate you. It would seem to make good trade sense for them to be more willing to trade a planet like that, or to take a similar planet of yours in trade. Close neighbors, of course, might feel very differently.

Do the human player's past actions figure in to trading at all? If you're a known warmonger, why would anyone even sell weapon techs to you? If you're a peaceful, good empire, other similar empires might take this into account.

Which brings me to my last point, computer player personality. What if the Iconians had a 'be kind to the galactic underdog' philosophy, or the Terrans were inclined to suck up to anyone with a higher military rating? This type of thing would be exploitable if it were always the same, but it would be nice to see some kind of race philosophy at work behind a computer player's diplomatic and trade styles - especially if it were random between games and the human player had to guess his way through it each time.

All of that would probably be almost impossible to factor in, but if you are serious about enhancing the computer players' trade/diplomatic functionality, I hope you will consider at least adding some situational awareness routines.
Reply #17 from GalCiv2 Forums Top
Wow, great thread!

My latest DA 1.6 Beta game, masochistic medium map as Thalans, the Terrans offered me Anti-matter Power Plants for Beam-Weapon Theory.

Needless to say, I took it.

Then the Drengin offered me Majesty for Stinger II.


So, once in a blue moon you'll get a good deal.
Reply #18 from GalCiv2 Forums Top
I'd have to agree. When you say that the Trade screen isn't involving any AI, that really points out why its such a hole in their playstyle.

Just giving it a quick thought... there's basically four parts to a trade that need to be considered on a trade item by trade item basis. It basically comes down to what each side is getting or losing with each side of the trade. Lets take a simple trade, like 500 BC for laser III. The loss of the laser would be considered differently for both sides... both sides would be thinking "what am I gaining/losing out of this, and what are THEY gaining/losing?" This stuff would HAVE to be hooked into the AI somehow...

How you make use of that information becomes highly variable. Trading done on good terms would probably be a net gain for both sides. Trading done on less... favourable terms would be more one sided, or even set up to give the other a raw deal at the expense of ones self.
Reply #19 from GalCiv2 Forums Top

Treaties should be bilateral only, the above screenshot is why.


But, but, I like my "tributes". The thing is, it wouldn't be an exploitable advantage if the AI just tried to do it as well. I think the current "free" treaty system can work, where some empires get six sent there way, others get none. But for it to work, the AI has to actually want them.


I agree, but I do think it's the easier fix. The current system *can* work, but I'm not sure it should. Considering the diplomatic bonuses provided by treaties the fact that you can have them with everyone just serves to make the diplo model all that much easier to exploit. I'd certainly love to see the AI manipulate that system as well, but I don't have any real expectations in the regard with GC2 (here's hoping that it can be worked out in GC3). It just seems like too massive a project for Stardock to be able to realistically manage in patches.
Reply #20 from GalCiv2 Forums Top
Maybe not patches, but an expansion that did an overhaul of the AI, including making trades actually work?

I'd pay money for that almost as fast as I would for multiplayer. Put the two together and you have the recipie for the perfect expansion - makes everyone happy, box it up and stick it in stores.

but now I'm off on a tangent so I'll get back to the discussion, sorry.
--------

The trade system really, really needs an overhaul - AIs just arent INTELLIGENT at all about trades, as many posters here have said. I'd also like to add another point to this bit about "stupid AI tricks". When you offer a treaty to give an AI some money to go beat up another race, the AI doesnt consider anything but relative military strength. Last game the Korx were at war with 3 civs and I paid them a scant 1500 to go to war with the other remaining AI civ. Because this final war overextended it, the Korx were overwhelmed by the might of four enemy AIs despite having the current military lead.

AIs need to evaluate the wars they and the target are involved in. It should cost substantially more, or even return a "our military is too busy in our other wars" response, to convince an AI to go aggressing more enemies when it's beset on all sides by races who want to destroy it.
Reply #21 from GalCiv2 Forums Top
I really really really hope you read this. I think you guys did an innovative, creative, and amazing job creating a trade system. I would feel proud of yourselves, because no one has done anything similar to what you did. You created a 5 fold trading system, and although there is always going to be a way to cheat any system or AI. You did something that was fun. Don't fall off your pedestal on the trade screen, but pat your own backs. Firaxis with their multimillion dollar budget didn't think of a trade screen anymore complex for CIV4 then the one they used for CIV3. Let alone the fact that the only improvements they added for their multimillion dollar expansion pack was 2 civs and 2 units. You guys revamped your trade screen to be 30 times more pimp for less of a budget. Thanks guys.
Reply #22 from GalCiv2 Forums Top
I very much disagree with the people who say the trading system isn't intelligent. You not only have an a great variety of trade goods, but you also have the diplomatic ability and negotian abilities which increase your chances of getting a good deal. That is creative fellows. I have only ran into two instances where the AI acted wierd on a trade, and that was on the lower levels. The AI bought a technology for a colonization for a world already possessed by someone. It happened to conquer that world later though. Plus I have seen the AI try trading smaller technologies that don't have much significance to me, but I try and trade useless stuff all the time. I think it was to raise the relation meter or because they needed the technology. I think people are upset that it's so hard to trade with the other races unless your human, and have an amazing diplomacy level. Thats the way it is in real life though. I don't see China trading nuclear secrets for canned good, and I don't see any country trading important technology for any amount of money unless a 'SPY' (which happens to be in your game) steals it. Another factor that is bothering some people, but I HAPPEN TO TRULY ENJOY THIS PART OF THE GAME is that civilizations with a good alignment are unwilling to trade weapons to evil civilizations, and vice versa. I know you can do that in CIV4, but I don't think that is truly realistic for advanced civilizations in space let alone the modern world. However if there is a way to revamp it please do, but I do enjoy the difficulty level of the AI TRADING.
Reply #23 from GalCiv2 Forums Top
Maybe not patches, but an expansion that did an overhaul of the AI, including making trades actually work?

I'd pay money for that almost as fast as I would for multiplayer. Put the two together and you have the recipie for the perfect expansion


Actually put those two together and you have one of the few expansions I wouldn't pay for for this game. I don't think the trading is that bad, and multiplayer is fundamentally useless to me. Incidentally there's no trading AI to overhaul, did you read the initial post here?
Reply #24 from JoeUser Forums Top
One other thought: When you make an offer, and the AI dialog box turns "Green", lowering your offer ought to make it turn "Red", no matter what else it would have accepted previously...
Reply #25 from GalCiv2 Forums Top
Incidentally there's no trading AI to overhaul

Which probably is the problem right there. I also agree, I don't think trading is that bad, and the 1.5x3 patch made the AI a little tougher to beat.

I do think trading is one of the biggest AI holes, although probably not fair to attribute it to the AI. But, to be fair, even as one of the biggest holes it isn't that big at the top levels, and I'm becoming a fanboy, I just love everything about this game.

Doesn't mean feedback can't help it to improve!