You definitely need to use Dialogflow CX if you are certain that you are building a slot filling bot. In my view, the bar for what qualifies as a slot filling bot is fairly high:
Learn more here
The second situation where you definitely need to use Dialogflow CX is for telephony bots and voice bots in general (once more tooling is available). This is because CX is well suited for phone tree style bots. It is hard to fully explain why in an email, but I would recommend that you log in to Dialogflow CX and load the prebuilt bots into your CX console. Once you spend some time studying these prebuilt agents, it will become clearer why CX is well suited for such bots.
In addition, the existing tooling for Telephony bots and voice bots in Dialogflow ES is quite poor, so there is very little reason to use ES for voice bots.