Comments on empirical results

By BigStupidJellyfish | 2021-March-12

(Note, 2022-May-14: this page is largely replaced by The Problem with Damage Simulators.)

Thanks to Target Saving, it's easier than ever to conduct field tests with your echelons. When possible and practical, I like to make use of this in my articles. This page will hold some general comments on my approach & methods with this system.

A primary motivation: DPS simulator numbers are often unreliable and don't translate well into in-game performance.

Grenade skills cannot be properly accounted for. The number of targets hit, their remaining HP when hit, and so on will vary quite a bit per-combat. Simulators generally assume some configurable number of enemy links to be hit and multiply the grenade damage by that number - this is not how combat actually goes. One issue this leads to is over-valuing FP buffs from HG skills/tiles on grenadiers.
Many other skills are also not properly modeled in a simulator. AN-94 and X95, for example, will do quite a bit more or less damage depending enemy density, damage state, and so on (though I have some estimates on X95 here).

A non-exhaustive list of some other common dolls that work in ways a simulator won't be able to describe well, not counting all the grenadiers: Type88 mod, KSVK mod, NTW mod, Rico, R93 (to a lesser extent), Carcano M91/38 (45x damage is amazing, but often is overkill and inflates simulator numbers), M4A1 (cannon), ACR, C-93, El Clear (if you kite her), Contender (for non-boss combat), Python, Kord, and JS 9.

Damage early is usually more valuable than damage later. If you can kill the frontline Aegis before they shield, that's a lot less HP to work through. Similarly, dead enemies can't shoot back. Killing off dangerous enemies early can save a lot of damage taken. On the other hand, fights with long walks or a few trash mobs at the start (7-6 Dreamer, for example) don't need as much in the way of frontloaded damage. Getting a 20-second total damage number doesn't help you either way here.
Many skills offer unique benefits in certain situations. DPS simulators can't explain those scenarios or measure the value provided by these skills. A few examples:

If your echelon is expecting to take field-wide AoE damage, HS2000 may be critical to keep your DPS healthy. M950A mod offers a very unique speed boost/slow field that can be very hard to replace.
Many ARs have unique attributes that help in certain encounters - RFB targets backline, MDR can shield your frontline, G11/AN-94/AR-15/K2 have multi-hit skills that deplete Orthrus shields faster, and so on.
Python's stream of FP buffs works very well with AS Val at night, and her skill in general isn't fully implemented on any simulator.

Manual skill control and kiting is important in tougher fights. Outside of applying a fixed skill delay, this isn't something DPS simulators can account for.
Many battles have significant random elements. Your MGSG will do much differently against SF groups with Jaguars in the back depending on if those Jaguars get taken out quickly or not.
Taking less damage yourself is also important. A DPS simulator can't tell you how valuable +30% EVA on your offtank is, especially if it's being considered against different buffs.

Damage simulators are most useful when you want to compare very similar options. They are good at answering questions like:

Will G41 or ART556 do more damage in this echelon?
Which equipment combination on this doll gives the highest DPS in this echelon?
Considering some expected enemy evasion, will Warrior or Artillery provide more DPS here on average?
What MGs do I need to get 54+ armor on my shotguns for 9-4N Judge?
What approximate time can I expect on the data dummy with this MG setup?
Will placing SSG3000 such that her tile covers Mk23 or Px4 Storm give more DPS?

They will almost never be a useful number for evaluating an echelon as a whole.

Field tests aren't perfect either. If trying to look into broad ideas or general cases, it's hard to say that any specific set of fights tested will perfectly represent the bigger picture. Each battle may also have certain mechanics that significantly change the numbers you get (e.g., Cyclops grenades deal a lot of unavoidable damage after a certain period). Kiting is a critical part of high-end fights, and consistently incorporating that into tests is challenging. Target practice starts with everything at full health, so you can't see how things turn out several fights in, and strategic options are limited (para debuff/SF chips/combo/construction/suee). They can help build your understanding of how different options will perform in a range of circumstances, but are hardly certain proof.

A few comments on other bits of methodology:

Test fights are generally performed with no kiting (for consistency purposes), so the teams tested should be able to handle the fight like that. Doppelsoldners, for example, usually need HG kiting & retreating to fight properly and don't work as well here. Hydras either get killed fast enough (resulting in no damage taken) or absolutely destroy your dolls (not giving much good information).
To avoid fairy talent RNG the fights are done either with 5* fairies, no fairies, or a fairy with a flex/useless talent (e.g., Armor in non-SG echelon). Fairy selection varies to modify the difficulty.
For the most part, all dolls used are maxed out (all relevant stats/best equipment/SL10/SPEQs). Any deviations from this, mostly in skill levels, will be noted as appropriate.
The echelons built are not necessarily optimal for the situation or even in general.
I try to take at least 10-20 trials per configuration. More is always better, but I don't have infinite time, there are usually many different possible options to test, nor do I make any money off this, so it'll have to do.
Force shields, mixed armor/non-armor without AP, varying evasion, and other such factors will make damage statistics less reliable. One doll shooting only half-shielded Gunners will deal less damage on paper than another shooting only unshielded Strelets, all else being equal. It's harder to rack up damage against a bunch of scouts compared to guards. When significant, shield HP depleted is added to each doll's damage dealt.