The reason flat damage boost is mostly useless aside of a few gimmicky fights is because it is, well, a flat damage boost. When you start out, it can be theoretically useful; if your servants are doing 2-3k damage with their face cards against enemies with 30-50k health (read: bosses in Orleans), adding an extra 500 flat damage on top of that is a sizable increase netting you at least 1500 extra damage per turn. However, when you are buff-stacking and your face-cards are doing 10k damage (or, during events with damage CEs, 30k+), that 500 extra damage becomes statistical noise.
I really wish flat damage boosts were per-hit instead of per-card. That way, a servant with a 5 hit quick card would suddenly deal 2.5k extra damage, which is no longer negligible, and for servants with 10 hit NPs, it would actually help hitting some damage benchmarks much easier.