There's only 2 fights so far that I found difficult without save-scumming the crap out of them:
The latter orc fights during the gathering: the rouge is VERY hard to hit twice reliably and even if you DO, you still need the ones AFTER him to either miss or do minimum damage to you to in order to even be able to drink the huge health potion. Buying 1 orc potion or hell, even a large health potion during Aka's route solves this. You can sell 10 of the regular health potions to cover the cost of a huge one. There is VERY little point in hoarding all the small potions, there are multiple points in the game that say so too ("Hoarding has been the death of many an adventurer", "They all said the were saving them for the 'final fight'", Antarion carrying like a million of the things, etc.)
The other one is Simon's solo fight with Mestan in Eustrin. Without a durge or sylvan sword, it's actually possible to lose.