asthasr

Join Date 02/2008

Practical Beta Suggestion

For the Betas, I think it would be a good idea if you implemented "postgame dumps." Essentially, these would be XML files containing information on technologies researched, spells used, items bought and the like. They could be parsed to provide statistics: individual technologies could be correlated with final scores, short/long games, and the like. If 95% of winners are taking, say, Earth Magic, this would provide valuable balancing information instead of relying on anecdotes.

It would probably be most valuable if submission was automated. Upon quitting a game, surrendering, winning or losing, the file could be submitted to a parsing service which adds it to a database. This wouldn't be acceptable for the retail release, obviously, but it could be part of the EULA for beta testers.

Locked Post

Advanced Search

Watch this post

Do not email me updates for this post Email me updates for this post

Successfully updated karma reason!

Sounds like a great idea.

Successfully updated karma reason!

Sounds like a good suggestion.

Successfully updated karma reason!

I must agree, this is a most potent suggestion. Statistical data on how people play and how they win is very important in balancing. It would be very easy to see if no one did some particular thing, proving it to be useless, or if everyone did some single thing, making it overpowered.

More than that, perhaps there should be a more detailed submission form you can make after winning or losing the game, where you could note certain things and put an urgency level to it, even to such a level of precision that you could tag certain technologies with tags based on your opinion, for exmple "Vital" for those you think are pretty much needed to even begin to win the game, "overpowered" to those you think are broken, "worthless" to those you think are rubbish, and so on. Such info could be gathered statistically as well, giving a more accurate report on the players beliefs of the power of certain techs, buildings, spells and whatever. If there's a lot of differing opinoins about something, like 30% of the people thinking something is overpowered and 70% that it's worthless, surely more intensive studying is needed. But then, I suppose if everyone who played a game submitted a report for every single game, the amount of stuff sent would be quite enormous.

Successfully updated karma reason!

the amount of stuff sent would be quite enormous.

From a dev point of view, an enormous amount of data would be a good thing. The only problem I see is that creating such a survey would probably require a lot of dev time, which is not cheap.

Successfully updated karma reason!

I don't think we should worry about postgame dumps, because some of the opponents are going to be bots on Stardock's servers. So if you can identify when the game was, and against which bots it was played, then they should be keeping track of the dumps on their end. Its a good idea, but I think stardock has an even better way of handling it.

Successfully updated karma reason!

Except if you were doing single player.

Successfully updated karma reason!

As far as the balancing goes - I think there should be some "not so great" options. Part of the fun of strategy games like this is (at least for me) learning the optimum path to victory. If all paths are balanced to pure equality, then choices can be neutered to meaninglessness.

Of course, every FACTION should balance against the others... but anyway...

As far as massive volume of input, the whole point of having it generate XML is so it could be input and summarized automatically on the back end. It really should not take much time to whip something up to do this and in the long run would be a huge time saver. Small investment up front would reap big benefits down the road.

I would be surprised if they don't already have tools that do this stuff.

Successfully updated karma reason!

The thing is, most games have techs or buildins or spells or whatever that are NORMALLY better than the others, but the difference comes when your situation is different from normal. Suddenly, a useless tech like Mountaineering will be great if most of your lands are mountains, or a normally super-powerful tech like Sailing is useless. In each game the player should "learn the optimum path to victory" as it is, since with variance, you'll have to figure out each time what's the best path this time. This is also exactly the reason why there shouldn't be any "standard starting positions". Sometimes you start near gold and wheat, sometimes you start in a rotten marsh. What matters is how you survive, despite your starting position.

Successfully updated karma reason!

Quoting Tiavals,

reply 8

I vehemently disagree on the "not so great" part. It would be quite ludicrous if there was a single best way of winning. Once you have it, the game would be dreadfully boring and meaningless, since you don't even make a choice because you're forced to take the best path.

The thing is, most games have techs or buildins or spells or whatever that are NORMALLY better than the others, but the difference comes when your situation is different from normal. Suddenly, a useless tech like Mountaineering will be great if most of your lands are mountains, or a normally super-powerful tech like Sailing is useless. In each game the player should "learn the optimum path to victory" as it is, since with variance, you'll have to figure out each time what's the best path this time. This is also exactly the reason why there shouldn't be any "standard starting positions". Sometimes you start near gold and wheat, sometimes you start in a rotten marsh. What matters is how you survive, despite your starting position.

I think you are misunderstanding me - there should be MANY optimal paths to victory. There should ALSO be many "not so great" options - part of playing a strategy game like this is going down some bad paths and learning which options don't really work. Better yet, what might not work for one person, maybe someone else will figure out some unique way to use that weak ability in a way that it will be the cornerstone for his victory.

As I read your second paragraph, I really think we are in agreement. I completely agree that there should be enough variety and randomness in the game that what workes well in one game may not work out at all in another. Being able to make adjustments for "the hand you are dealt" definitely needs to be more important than determining an optimum tech path to always win.

Successfully updated karma reason!

It's not a question of finding the "slightly sub-optimal" paths and weeding those out. Those are the lifeblood of an interesting gaming experience. Instead, think about a tech which occurs in 90% of won games. This is a tech which is dangerously close to "necessary." It might simply be foundational, like researching "Alphabet" in Civ IV, but it might also be a truly off-the-wall spell that is simply too strong. This can occur even without the players realizing that it's happening, and statistical awareness lets you see past some of the "self-blindness" that people suffer when analyzing their own actions.

Successfully updated karma reason!

Stat collection is always fine; that is, it's never bad to have tons and tons of stats. However, how the devs interpret them and which stats to weigh against other stats is a very tricky subject. Your example of the 90% situation is actually a great one. That stat in of itself doesn't actually say much. Is it necessary to get to another tech down the line that's great? Is it the tech that leads to the most flexible tech path so you can counter your opponent? Is the tech itself overpowered? Furthermore, is a necessary tech necessarily a bad thing (think rank fire in E:TW)?

What ends up happening is that you need the subjective input of the player in the end anyway and hope that they, as beta testers, can be as detailed as possible in their own self-analysis.

Successfully updated karma reason!

Well, interpretation is always the problem with statistics, DeCypher. There are a few possibilities in the example I gave:

The tech itself may be imbalanced.
A descendant tech may be imbalanced.
The tech may be necessary to access a flexible branch of the techology tree.
The imbalance may exist, but be intentional.

Case 1 should be easy to diagnose. It would exhibit itself as a tech with a high selection rate across several hundred games followed by descendant techs with statistically acceptable selection rates.

Case 2 will show a string of imbalanced techs until a final tech, at which point it should revert to Case 1.

Case 3 is probably the most problematic case, because it means a large portion of the content is going unused.

Case 4 is often bad game design if it occurs at an early stage. Presuming people are expected to take a certain tech at a much higher rate than its alternatives, why spend time on the alternatives at all? Development resources are finite.

Consider a hypothetical tech tree:

A (65%)
- Aa (75%)
  - Aa1 (50%)
  - Aa2 (50%)
- Ab (25%)
  - Ab1 (50%)
  - Ab2 (50%)
B (35%)
- Ba (90%)
  - Ba1 (70%)
  - Ba2 (30%)
- Bb (10%)
  - Bb1 (100%)
  - Bb2 (0%)

There are a few problems here. A vs. B reveals a problem like Case 4. The B branch is being neglected in favor of the A branch. In the second tier of technolgoies, Aa is individually overpowered; the Aa1 and Aa2 descendant techs are both used evenly. Aa may be the tech which is causing the B branch to be neglected. Similarly, Ba is grossly overpowered compared to Bb, but doesn't seem to be the "terminal" option--Ba1 is also overused compared to Ba2. At a later tech tier like this, the 7:3 ratio is more acceptable than at the foundational tier, however, so this may be desired behavior in which Ba2 serves a special purpose... but Ba vs. Bb is still problematic.

Successfully updated karma reason!

See, that's the type of behavior I do NOT want the devs to employ. For all we know, Ba and Bb are exactly the same in terms of power, but Ba looks cooler.

You can't tell how overpowered something is simply by looking at how popular it is.

Successfully updated karma reason!

It's never going to go 9:1 because of a cooler effect. That said, even in the case of something like that, the skills should be tweaked. If Ba is being used 9:1 that means that Bb, Bb1, and Bb2 were wasted development time. That has to be weighed by the devs, of course. This is not intended to be an automatic, emergent system. Think about it like this: if the statistics I posted are based on 1,000 games, that means that these techs were chosen...

A (650 times)
- Aa (488 times)
  - Aa1 (244 times)
  - Aa2 (244 times)
- Ab (163 times)
  - Ab1 (81 times)
  - Ab2 (81 times)
B (350 times)
- Ba (315 times)
  - Ba1 (221 times)
  - Ba2 (95 times)
- Bb (35 times)
  - Bb1 (35 times)
  - Bb2 (0 times)

If it's arranged like this, the problem may be more obvious. There are exactly three paths in this pile of stats that are getting used: A->Aa->Aa1, A->Aa->Aa2, and B->Ba->Ba1. Together they account for 70% of the techs chosen. Presumably the people who designed and implemented the other five would like for them to be used, as well.

Successfully updated karma reason!

Yes, I totally hear you on this. There was one game (can't remember) that a virtually unbeatable strategy was "race to tech x, disregarding everything else except for turtling up to that point. Then spew out as many of unit y as possible that is now incredible because of above tech x and zerg the hell out of the AI.

IT can be fun to "discover" this path, but it usually means that I will probably play the game 2 naybe 3 times after that. (Unless the gameplay is just so fun that I like to cripple myself just for the challenge)

Successfully updated karma reason!